[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:16 dist-test-slave-3kk6 krb5kdc[17334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:16 dist-test-slave-3kk6 krb5kdc[17334](info): set up 2 sockets
Aug 06 22:37:16 dist-test-slave-3kk6 krb5kdc[17334](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 22:37:17.429668 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.393s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:17 dist-test-slave-3kk6 krb5kdc[17334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:17.565567 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:35581
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:35581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:17.674917 17353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:17.675238 17353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:17.675290 17353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:17.676561 17353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:17.676641 17353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:17.676667 17353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:17.676689 17353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:17.676728 17353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:17.679284 17353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:35581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:35581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:17.679790 17353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:17.680102 17353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:17.687135 17360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:17.687085 17362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:17.687156 17353 server_base.cc:1034] running on GCE node
W20260806 22:37:17.688230 17359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:17.688679 17353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:17.689128 17353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:17.690299 17353 hybrid_clock.cc:648] HybridClock initialized: now 1786055837690266 us; error 87 us; skew 500 ppm
Aug 06 22:37:17 dist-test-slave-3kk6 krb5kdc[17334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055837, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:17.693859 17353 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:17.694599 17353 webserver.cc:460] Webserver started at http://127.16.235.126:40893/ using document root <none> and password file <none>
I20260806 22:37:17.694942 17353 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:17.694998 17353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:17.695142 17353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:17.696414 17353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "dde5c76867b54cd6bedda9b3de24da27"
format_stamp: "Formatted at 2026-08-06 22:37:17 on dist-test-slave-3kk6"
server_key: "713ab04f59941f35f3711cac237ba11c"
server_key_iv: "76f2ab7348a2ec6b567b4835d1e6ea81"
server_key_version: "encryptionkey@0"
I20260806 22:37:17.696785 17353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "dde5c76867b54cd6bedda9b3de24da27"
format_stamp: "Formatted at 2026-08-06 22:37:17 on dist-test-slave-3kk6"
server_key: "713ab04f59941f35f3711cac237ba11c"
server_key_iv: "76f2ab7348a2ec6b567b4835d1e6ea81"
server_key_version: "encryptionkey@0"
I20260806 22:37:17.698820 17353 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260806 22:37:17.699985 17371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:17.700304 17353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:17.700424 17353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "dde5c76867b54cd6bedda9b3de24da27"
format_stamp: "Formatted at 2026-08-06 22:37:17 on dist-test-slave-3kk6"
server_key: "713ab04f59941f35f3711cac237ba11c"
server_key_iv: "76f2ab7348a2ec6b567b4835d1e6ea81"
server_key_version: "encryptionkey@0"
I20260806 22:37:17.700512 17353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:17.721860 17353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:17.724884 17353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:17.725229 17353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:17.732365 17353 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:35581
I20260806 22:37:17.732462 17444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:35581 every 8 connection(s)
I20260806 22:37:17.733080 17353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:17.734555 17445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:17.738368 17445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27: Bootstrap starting.
I20260806 22:37:17.739667 17445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:17.740020 17445 log.cc:824] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:17.740991 17445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27: No bootstrap required, opened a new log
I20260806 22:37:17.743059 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 17353
I20260806 22:37:17.743191 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:17.743110 17445 raft_consensus.cc:348] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde5c76867b54cd6bedda9b3de24da27" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35581 } }
I20260806 22:37:17.743371 17445 raft_consensus.cc:374] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:17.743466 17445 raft_consensus.cc:729] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dde5c76867b54cd6bedda9b3de24da27, State: Initialized, Role: FOLLOWER
I20260806 22:37:17.743652 17445 consensus_queue.cc:260] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde5c76867b54cd6bedda9b3de24da27" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35581 } }
I20260806 22:37:17.743817 17445 raft_consensus.cc:388] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:17.743856 17445 raft_consensus.cc:482] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:17.743911 17445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:17.744108 17325 external_mini_cluster.cc:1428] Setting key 5b109a6573be351fd95b368609518b36
I20260806 22:37:17.744973 17445 raft_consensus.cc:504] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde5c76867b54cd6bedda9b3de24da27" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35581 } }
I20260806 22:37:17.745183 17445 leader_election.cc:302] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dde5c76867b54cd6bedda9b3de24da27; no voters:
I20260806 22:37:17.745472 17445 leader_election.cc:288] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:17.745496 17449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:17.745798 17449 raft_consensus.cc:686] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [term 1 LEADER]: Becoming Leader. State: Replica: dde5c76867b54cd6bedda9b3de24da27, State: Running, Role: LEADER
I20260806 22:37:17.746035 17449 consensus_queue.cc:237] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde5c76867b54cd6bedda9b3de24da27" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35581 } }
I20260806 22:37:17.746021 17445 sys_catalog.cc:564] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:17.746940 17450 sys_catalog.cc:455] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dde5c76867b54cd6bedda9b3de24da27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde5c76867b54cd6bedda9b3de24da27" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35581 } } }
I20260806 22:37:17.747198 17450 sys_catalog.cc:458] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:17.747099 17451 sys_catalog.cc:455] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dde5c76867b54cd6bedda9b3de24da27. Latest consensus state: current_term: 1 leader_uuid: "dde5c76867b54cd6bedda9b3de24da27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dde5c76867b54cd6bedda9b3de24da27" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35581 } } }
I20260806 22:37:17.747490 17451 sys_catalog.cc:458] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:17.747761 17457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:17.749234 17457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:17.752399 17457 catalog_manager.cc:1347] Generated new cluster ID: a01c2cba567d470194418139eecfc050
I20260806 22:37:17.752458 17457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:37:17 dist-test-slave-3kk6 krb5kdc[17334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:17.763118 17457 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:17.763798 17457 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:17.767633 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:17.745764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52400 (local address 127.16.235.126:35581)
0806 22:37:17.746383 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:17.746391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:17.747837 (+ 1446us) server_negotiation.cc:407] Connection header received
0806 22:37:17.750998 (+ 3161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:17.751005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:17.751126 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:17.751287 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:17.753618 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:17.755328 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:17.757022 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:17.757718 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:17.757844 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:17.762849 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:17.762854 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:17.762854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:17.762869 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:17.765302 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:17.765880 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:17.765883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:17.765884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:17.765992 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:17.766318 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:17.766319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:17.766319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:17.766579 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:17.766803 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:17.767101 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:37:17.767305 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":135,"threads_started":1}
I20260806 22:37:17.777087 17457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dde5c76867b54cd6bedda9b3de24da27: Generated new TSK 0
I20260806 22:37:17.777314 17457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:17.814409 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35581
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:17.900249 17480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:17.900511 17480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:17.900555 17480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:17.901731 17480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:17.901808 17480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:17.901856 17480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:17.903573 17480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35581
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:17.903939 17480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:17.904197 17480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:17.912505 17491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:17.912575 17488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:17.912628 17494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:17.912791 17480 server_base.cc:1034] running on GCE node
I20260806 22:37:17.913019 17480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:17.913306 17480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:17.914513 17480 hybrid_clock.cc:648] HybridClock initialized: now 1786055837914496 us; error 35 us; skew 500 ppm
Aug 06 22:37:17 dist-test-slave-3kk6 krb5kdc[17334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055837, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:17.918532 17480 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:17.919167 17480 webserver.cc:460] Webserver started at http://127.16.235.65:37875/ using document root <none> and password file <none>
I20260806 22:37:17.919438 17480 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:17.919495 17480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:17.919622 17480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:17.921221 17480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "c000f11b5f6740d8ac6101ba25a806eb"
format_stamp: "Formatted at 2026-08-06 22:37:17 on dist-test-slave-3kk6"
server_key: "a2405a342e238163a77a32a1d1812429"
server_key_iv: "ea6b8ad3dc001cb1ab703706eee30181"
server_key_version: "encryptionkey@0"
I20260806 22:37:17.921679 17480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "c000f11b5f6740d8ac6101ba25a806eb"
format_stamp: "Formatted at 2026-08-06 22:37:17 on dist-test-slave-3kk6"
server_key: "a2405a342e238163a77a32a1d1812429"
server_key_iv: "ea6b8ad3dc001cb1ab703706eee30181"
server_key_version: "encryptionkey@0"
I20260806 22:37:17.923761 17480 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260806 22:37:17.924700 17503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:17.924901 17480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:17.925016 17480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "c000f11b5f6740d8ac6101ba25a806eb"
format_stamp: "Formatted at 2026-08-06 22:37:17 on dist-test-slave-3kk6"
server_key: "a2405a342e238163a77a32a1d1812429"
server_key_iv: "ea6b8ad3dc001cb1ab703706eee30181"
server_key_version: "encryptionkey@0"
I20260806 22:37:17.925113 17480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:17.935475 17480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:17.937628 17480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:17.937830 17480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:17.938175 17480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:17.938632 17480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:17.938786 17480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:17.938817 17480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:17.938830 17480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:17.948371 17480 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:40563
I20260806 22:37:17.948431 17656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:40563 every 8 connection(s)
I20260806 22:37:17.948858 17480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:17.951121 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 17480
I20260806 22:37:17.951236 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:17.951388 17325 external_mini_cluster.cc:1428] Setting key 886a701e0409ab498d50188bfbab0e03
Aug 06 22:37:17 dist-test-slave-3kk6 krb5kdc[17334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055837, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:17.960265 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:17.950029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:42023 (local address 127.16.235.126:35581)
0806 22:37:17.950082 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:17.950083 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:17.950317 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:37:17.950959 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:17.950961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:17.950975 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:17.951055 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:17.952456 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:17.952991 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:17.954047 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:17.954254 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:17.954331 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:17.956784 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:17.956786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:17.956786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:17.956795 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:17.958608 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:17.959196 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:17.959197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:17.959197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:17.959281 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:17.959539 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:17.959540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:17.959540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:17.959700 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:17.959779 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:17.960048 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 22:37:17.960152 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:37:17.960294 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:17.949989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:35581 (local address 127.16.235.65:42023)
0806 22:37:17.950174 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:17.950215 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:17.950843 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:17.951141 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:17.951145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:17.952090 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:17.952354 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:17.952360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:17.953099 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:17.953100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:17.953939 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:17.953941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:17.954342 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:17.954344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:17.954481 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:17.955172 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:17.955181 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:17.956686 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:17.958869 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:17.958876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:17.958878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:17.959112 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:17.959362 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:17.959363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:17.959363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:17.959468 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:17.959774 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:17.959779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:17.959903 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 22:37:17.959990 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:37:17.960119 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260806 22:37:17.960928 17657 heartbeater.cc:347] Connected to a master server at 127.16.235.126:35581
I20260806 22:37:17.961045 17657 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:17.961256 17657 heartbeater.cc:511] Master 127.16.235.126:35581 requested a full tablet report, sending...
I20260806 22:37:17.962349 17389 ts_manager.cc:194] Registered new tserver with Master: c000f11b5f6740d8ac6101ba25a806eb (127.16.235.65:40563)
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
I20260806 22:37:17.963531 17389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:42023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:17.983134 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35581
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:18.087379 17663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:18.087742 17663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:18.087788 17663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:18.089144 17663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:18.089196 17663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:18.089232 17663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:18.091135 17663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35581
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:18.091440 17663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:18.091741 17663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:18.098363 17669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:18.098435 17668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:18.098431 17671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:18.098948 17663 server_base.cc:1034] running on GCE node
I20260806 22:37:18.099227 17663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:18.099550 17663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:18.100780 17663 hybrid_clock.cc:648] HybridClock initialized: now 1786055838100744 us; error 69 us; skew 500 ppm
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[17334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:18.105448 17663 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:18.106204 17663 webserver.cc:460] Webserver started at http://127.16.235.66:36041/ using document root <none> and password file <none>
I20260806 22:37:18.106513 17663 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:18.106585 17663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:18.106839 17663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:18.108242 17663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "54b38c2d7537466d9b38ff17163b186e"
format_stamp: "Formatted at 2026-08-06 22:37:18 on dist-test-slave-3kk6"
server_key: "fad43bc731eef10d664c0e2edf94dfde"
server_key_iv: "fcd67ca3e49e17ea05e21a4381bb7200"
server_key_version: "encryptionkey@0"
I20260806 22:37:18.108665 17663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "54b38c2d7537466d9b38ff17163b186e"
format_stamp: "Formatted at 2026-08-06 22:37:18 on dist-test-slave-3kk6"
server_key: "fad43bc731eef10d664c0e2edf94dfde"
server_key_iv: "fcd67ca3e49e17ea05e21a4381bb7200"
server_key_version: "encryptionkey@0"
I20260806 22:37:18.110487 17663 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:37:18.111652 17684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:18.111893 17663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:18.112004 17663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "54b38c2d7537466d9b38ff17163b186e"
format_stamp: "Formatted at 2026-08-06 22:37:18 on dist-test-slave-3kk6"
server_key: "fad43bc731eef10d664c0e2edf94dfde"
server_key_iv: "fcd67ca3e49e17ea05e21a4381bb7200"
server_key_version: "encryptionkey@0"
I20260806 22:37:18.112095 17663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:18.125113 17663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:18.128261 17663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:18.128523 17663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:18.128949 17663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:18.129405 17663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:18.129475 17663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:18.129519 17663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:18.129547 17663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:18.142962 17663 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:35799
I20260806 22:37:18.142998 17841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:35799 every 8 connection(s)
I20260806 22:37:18.143402 17663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:18.150098 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 17663
I20260806 22:37:18.150194 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:18.150411 17325 external_mini_cluster.cc:1428] Setting key d0fe11ed1bc4db274c662404f5bef5f4
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[17334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:18.155795 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:18.144450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:56479 (local address 127.16.235.126:35581)
0806 22:37:18.144497 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:18.144499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:18.144722 (+ 223us) server_negotiation.cc:407] Connection header received
0806 22:37:18.145486 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:18.145487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:18.145497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:18.145564 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:18.146720 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.147305 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.148474 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.148662 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.148733 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:18.152085 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:18.152087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:18.152088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:18.152099 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:18.154064 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:18.154640 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:18.154641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:18.154641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:18.154748 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:18.155056 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:18.155058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:18.155058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:18.155206 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:18.155287 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:18.155591 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 22:37:18.155699 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:37:18.156032 17844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:18.144407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:35581 (local address 127.16.235.66:56479)
0806 22:37:18.144578 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:18.144624 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:18.145383 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:18.145645 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:18.145649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:18.146331 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:18.146546 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:18.146552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.147407 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.147408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:18.148384 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:18.148387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.148780 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.148781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:18.148925 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:18.149599 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:18.149609 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:18.151974 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:18.154272 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:18.154281 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:18.154283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:18.154556 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:18.154849 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:18.154850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:18.154850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:18.154971 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:18.155291 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:18.155299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:18.155449 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 22:37:18.155566 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:37:18.155748 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260806 22:37:18.156850 17842 heartbeater.cc:347] Connected to a master server at 127.16.235.126:35581
I20260806 22:37:18.156957 17842 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:18.157181 17842 heartbeater.cc:511] Master 127.16.235.126:35581 requested a full tablet report, sending...
I20260806 22:37:18.157649 17389 ts_manager.cc:194] Registered new tserver with Master: 54b38c2d7537466d9b38ff17163b186e (127.16.235.66:35799)
I20260806 22:37:18.158068 17389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:56479
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:18.188014 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35581
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:18.283576 17849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:18.283845 17849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:18.283892 17849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:18.285092 17849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:18.285166 17849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:18.285214 17849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:18.286831 17849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35581
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:18.287117 17849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:18.287361 17849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:18.292773 17858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:18.292773 17861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:18.292805 17857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:18.292961 17849 server_base.cc:1034] running on GCE node
I20260806 22:37:18.293309 17849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:18.293533 17849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:18.294708 17849 hybrid_clock.cc:648] HybridClock initialized: now 1786055838294694 us; error 34 us; skew 500 ppm
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[17334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:18.300715 17849 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:18.301312 17849 webserver.cc:460] Webserver started at http://127.16.235.67:36191/ using document root <none> and password file <none>
I20260806 22:37:18.301568 17849 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:18.301641 17849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:18.301805 17849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:18.303215 17849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "786826650d854f8ba45f7c641be1be0a"
format_stamp: "Formatted at 2026-08-06 22:37:18 on dist-test-slave-3kk6"
server_key: "8cc43c200ad7360e3825b520b9e29757"
server_key_iv: "822f7fe8fd209d3904b2af5d92072628"
server_key_version: "encryptionkey@0"
I20260806 22:37:18.303678 17849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "786826650d854f8ba45f7c641be1be0a"
format_stamp: "Formatted at 2026-08-06 22:37:18 on dist-test-slave-3kk6"
server_key: "8cc43c200ad7360e3825b520b9e29757"
server_key_iv: "822f7fe8fd209d3904b2af5d92072628"
server_key_version: "encryptionkey@0"
I20260806 22:37:18.305369 17849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:18.306281 17869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:18.306495 17849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:18.306627 17849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "786826650d854f8ba45f7c641be1be0a"
format_stamp: "Formatted at 2026-08-06 22:37:18 on dist-test-slave-3kk6"
server_key: "8cc43c200ad7360e3825b520b9e29757"
server_key_iv: "822f7fe8fd209d3904b2af5d92072628"
server_key_version: "encryptionkey@0"
I20260806 22:37:18.306895 17849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:18.323551 17849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:18.326084 17849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:18.326280 17849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:18.326645 17849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:18.327105 17849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:18.327168 17849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:18.327216 17849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:18.327244 17849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:18.336164 17849 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:43999
I20260806 22:37:18.336210 18034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:43999 every 8 connection(s)
I20260806 22:37:18.336602 17849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[17334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:18.345055 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 17849
I20260806 22:37:18.345166 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:18.345368 17325 external_mini_cluster.cc:1428] Setting key a6ee160a20fd1c24120f9f0a93c8bd7d
I20260806 22:37:18.348084 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:18.337674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:36815 (local address 127.16.235.126:35581)
0806 22:37:18.337752 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:18.337754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:18.337992 (+ 238us) server_negotiation.cc:407] Connection header received
0806 22:37:18.338487 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:18.338488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:18.338500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:18.338558 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:18.339582 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.340187 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.341302 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.341537 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.341594 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:18.344382 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:18.344384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:18.344384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:18.344393 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:18.346395 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:18.346980 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:18.346981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:18.346982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:18.347058 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:18.347384 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:18.347385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:18.347385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:18.347589 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:18.347654 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:18.347938 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 22:37:18.348015 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:37:18.348263 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:18.337626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:35581 (local address 127.16.235.67:36815)
0806 22:37:18.337846 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:18.337886 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:18.338392 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:18.338615 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:18.338618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:18.339274 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:18.339492 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:18.339497 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.340288 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.340289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:18.341218 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:18.341220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.341652 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.341685 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:18.341823 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:18.342528 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:18.342538 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:18.344236 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:18.346504 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:18.346512 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:18.346514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:18.346902 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:18.347162 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:18.347163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:18.347163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:18.347293 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:18.347657 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:18.347663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:18.347792 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 22:37:18.347888 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:37:18.348027 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":138,"threads_started":1}
I20260806 22:37:18.348825 18036 heartbeater.cc:347] Connected to a master server at 127.16.235.126:35581
I20260806 22:37:18.348917 18036 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:18.349123 18036 heartbeater.cc:511] Master 127.16.235.126:35581 requested a full tablet report, sending...
I20260806 22:37:18.349555 17389 ts_manager.cc:194] Registered new tserver with Master: 786826650d854f8ba45f7c641be1be0a (127.16.235.67:43999)
I20260806 22:37:18.349977 17389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:36815
I20260806 22:37:18.358460 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[17334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[17334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:18.386193 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:18.377061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.16.235.65:40563)
0806 22:37:18.377358 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:18.377364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:18.377377 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:37:18.377417 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:18.377419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:18.377434 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:18.377511 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:18.378163 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.379084 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.379856 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.380228 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.380291 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:18.383168 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:18.383181 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:18.383181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:18.383189 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:18.384926 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:18.385375 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:18.385377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:18.385378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:18.385446 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:18.385677 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:18.385677 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:18.385678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:18.385833 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:18.385942 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:18.386038 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:37:18.386105 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":122,"threads_started":1}
W20260806 22:37:18.386741 17541 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47926
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[17334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[17334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:18.413349 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:18.405371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47940 (local address 127.16.235.65:40563)
0806 22:37:18.405461 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:18.405463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:18.405495 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:37:18.405577 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:18.405579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:18.405589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:18.405642 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:18.406262 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.407068 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.407792 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:18.408044 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:18.408099 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:18.410614 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:18.410616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:18.410617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:18.410628 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:18.412217 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:18.412549 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:18.412550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:18.412550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:18.412605 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:18.412783 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:18.412784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:18.412784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:18.412892 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:18.413086 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:18.413115 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 22:37:18.413183 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:37:18.414001 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 17480
I20260806 22:37:18.428611 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 17663
I20260806 22:37:18.439528 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 17849
I20260806 22:37:18.456084 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 17353
2026-08-06T22:37:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2453 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[18057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[18057](info): set up 2 sockets
Aug 06 22:37:18 dist-test-slave-3kk6 krb5kdc[18057](info): commencing operation
krb5kdc: starting...
W20260806 22:37:19.691435 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:19 dist-test-slave-3kk6 krb5kdc[18057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:19.833814 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40105
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:40105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:19.923132 18076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:19.923386 18076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:19.923413 18076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:19.924578 18076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:19.924623 18076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:19.924638 18076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:19.924651 18076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:19.924664 18076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:19.926155 18076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:40105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:19.926388 18076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:19.926594 18076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:19.932132 18082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:19.932132 18085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:19.932128 18083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:19.932538 18076 server_base.cc:1034] running on GCE node
I20260806 22:37:19.932852 18076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:19.933189 18076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:19.934384 18076 hybrid_clock.cc:648] HybridClock initialized: now 1786055839934369 us; error 50 us; skew 500 ppm
Aug 06 22:37:19 dist-test-slave-3kk6 krb5kdc[18057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055839, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:19.937342 18076 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:19.937908 18076 webserver.cc:460] Webserver started at http://127.16.235.126:41043/ using document root <none> and password file <none>
I20260806 22:37:19.938162 18076 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:19.938237 18076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:19.938413 18076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:19.939770 18076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "79a0cd882e8c4c669a3c481a872e344d"
format_stamp: "Formatted at 2026-08-06 22:37:19 on dist-test-slave-3kk6"
server_key: "b52d0ba652ab9641e7bf20abc0f0db01"
server_key_iv: "b28cdac2e0b2103670916e3afc877b41"
server_key_version: "encryptionkey@0"
I20260806 22:37:19.940212 18076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "79a0cd882e8c4c669a3c481a872e344d"
format_stamp: "Formatted at 2026-08-06 22:37:19 on dist-test-slave-3kk6"
server_key: "b52d0ba652ab9641e7bf20abc0f0db01"
server_key_iv: "b28cdac2e0b2103670916e3afc877b41"
server_key_version: "encryptionkey@0"
I20260806 22:37:19.941764 18076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:19.942607 18095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:19.942939 18076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:19.943058 18076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "79a0cd882e8c4c669a3c481a872e344d"
format_stamp: "Formatted at 2026-08-06 22:37:19 on dist-test-slave-3kk6"
server_key: "b52d0ba652ab9641e7bf20abc0f0db01"
server_key_iv: "b28cdac2e0b2103670916e3afc877b41"
server_key_version: "encryptionkey@0"
I20260806 22:37:19.943140 18076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:19.953840 18076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:19.956627 18076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:19.956881 18076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:19.961737 18076 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:40105
I20260806 22:37:19.961786 18171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:40105 every 8 connection(s)
I20260806 22:37:19.962216 18076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:19.963341 18172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:19.965994 18172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d: Bootstrap starting.
I20260806 22:37:19.966758 18172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:19.967092 18172 log.cc:824] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:19.967967 18172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d: No bootstrap required, opened a new log
I20260806 22:37:19.969674 18172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79a0cd882e8c4c669a3c481a872e344d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40105 } }
I20260806 22:37:19.969839 18172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:19.969868 18172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79a0cd882e8c4c669a3c481a872e344d, State: Initialized, Role: FOLLOWER
I20260806 22:37:19.970048 18172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79a0cd882e8c4c669a3c481a872e344d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40105 } }
I20260806 22:37:19.970129 18172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:19.970153 18172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:19.970206 18172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:19.970404 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 18076
I20260806 22:37:19.970510 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:19.970755 17325 external_mini_cluster.cc:1428] Setting key 9f07218c7881bc6bcd950a81eadaf12b
I20260806 22:37:19.971091 18172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79a0cd882e8c4c669a3c481a872e344d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40105 } }
I20260806 22:37:19.971232 18172 leader_election.cc:302] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79a0cd882e8c4c669a3c481a872e344d; no voters:
I20260806 22:37:19.971481 18172 leader_election.cc:288] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:19.971545 18178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:19.971737 18178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [term 1 LEADER]: Becoming Leader. State: Replica: 79a0cd882e8c4c669a3c481a872e344d, State: Running, Role: LEADER
I20260806 22:37:19.971793 18172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:19.971848 18178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79a0cd882e8c4c669a3c481a872e344d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40105 } }
I20260806 22:37:19.972201 18178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79a0cd882e8c4c669a3c481a872e344d. Latest consensus state: current_term: 1 leader_uuid: "79a0cd882e8c4c669a3c481a872e344d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79a0cd882e8c4c669a3c481a872e344d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40105 } } }
I20260806 22:37:19.972297 18178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:19.972512 18188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:19.972683 18179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79a0cd882e8c4c669a3c481a872e344d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79a0cd882e8c4c669a3c481a872e344d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40105 } } }
I20260806 22:37:19.972848 18179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:19.973410 18188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:19.975368 18188 catalog_manager.cc:1347] Generated new cluster ID: c3228c302f4647d79e88da45d85de5e7
I20260806 22:37:19.975435 18188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:37:19 dist-test-slave-3kk6 krb5kdc[18057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:19.980176 18188 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:19.980804 18188 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:19.984882 18188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79a0cd882e8c4c669a3c481a872e344d: Generated new TSK 0
I20260806 22:37:19.985064 18188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:37:19.985852 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:19.971352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.16.235.126:40105)
0806 22:37:19.971738 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:19.971749 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:19.971794 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:37:19.971929 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:19.971937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:19.971999 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:19.972159 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:19.973613 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:19.975095 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:19.976248 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:19.976924 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:19.977044 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:19.980369 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:19.980374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:19.980375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:19.980390 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:19.983738 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:19.984243 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:19.984247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:19.984247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:19.984376 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:19.984677 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:19.984679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:19.984679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:19.985023 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:19.985169 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:19.985459 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 22:37:19.985636 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:20.018533 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40105
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:20.112345 18207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:20.112600 18207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:20.112629 18207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:20.113915 18207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:20.113974 18207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:20.114005 18207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:20.115954 18207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:20.116263 18207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:20.116554 18207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:20.121990 18213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:20.122009 18217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:20.121976 18219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:20.122411 18207 server_base.cc:1034] running on GCE node
I20260806 22:37:20.122704 18207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:20.123041 18207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:20.124236 18207 hybrid_clock.cc:648] HybridClock initialized: now 1786055840124221 us; error 28 us; skew 500 ppm
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:20.127282 18207 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:20.127892 18207 webserver.cc:460] Webserver started at http://127.16.235.65:45505/ using document root <none> and password file <none>
I20260806 22:37:20.128180 18207 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:20.128259 18207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:20.128432 18207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:20.129593 18207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "c1cad0822ff045dba9206b682e97cac1"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-3kk6"
server_key: "fe49f3738de099108025db1e4c6ae0c8"
server_key_iv: "c8cfadf5353934c195b9bc2137da0855"
server_key_version: "encryptionkey@0"
I20260806 22:37:20.130007 18207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "c1cad0822ff045dba9206b682e97cac1"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-3kk6"
server_key: "fe49f3738de099108025db1e4c6ae0c8"
server_key_iv: "c8cfadf5353934c195b9bc2137da0855"
server_key_version: "encryptionkey@0"
I20260806 22:37:20.131592 18207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:20.132503 18227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:20.132732 18207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:20.132860 18207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "c1cad0822ff045dba9206b682e97cac1"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-3kk6"
server_key: "fe49f3738de099108025db1e4c6ae0c8"
server_key_iv: "c8cfadf5353934c195b9bc2137da0855"
server_key_version: "encryptionkey@0"
I20260806 22:37:20.132946 18207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:20.145646 18207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:20.148198 18207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:20.148389 18207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:20.148658 18207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:20.149008 18207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:20.149065 18207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:20.149118 18207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:20.149148 18207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:20.155979 18207 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:44771
I20260806 22:37:20.156024 18389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:44771 every 8 connection(s)
I20260806 22:37:20.156405 18207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:20.165740 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 18207
I20260806 22:37:20.165859 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:20.166045 17325 external_mini_cluster.cc:1428] Setting key d463d959a7cab33aaa0ff1346640cae2
I20260806 22:37:20.167692 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.157286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40105 (local address 127.16.235.65:43313)
0806 22:37:20.157486 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:20.157531 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:20.158805 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:20.159073 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:20.159076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:20.159379 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:20.159589 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.159593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.160293 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.160295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.161282 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.161286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.161715 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.161716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.161841 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.162409 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:20.162419 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:20.164002 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:20.166025 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:20.166039 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:20.166041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:20.166346 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:20.166561 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:20.166562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:20.166562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:20.166743 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:20.167099 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:20.167108 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:20.167250 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 22:37:20.167360 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:37:20.167507 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
I20260806 22:37:20.167690 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.157345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:43313 (local address 127.16.235.126:40105)
0806 22:37:20.157421 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.157424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.157626 (+ 202us) server_negotiation.cc:407] Connection header received
0806 22:37:20.158917 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.158919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.158934 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.158994 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:20.159670 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.160180 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.161379 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.161585 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.161646 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.164120 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:20.164122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:20.164123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:20.164131 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:20.165904 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.166408 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.166410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.166410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.166485 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.166825 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.166826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.166827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.167004 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:20.167085 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.167425 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.167544 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:37:20.168630 18390 heartbeater.cc:347] Connected to a master server at 127.16.235.126:40105
I20260806 22:37:20.168787 18390 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:20.169013 18390 heartbeater.cc:511] Master 127.16.235.126:40105 requested a full tablet report, sending...
I20260806 22:37:20.169643 18122 ts_manager.cc:194] Registered new tserver with Master: c1cad0822ff045dba9206b682e97cac1 (127.16.235.65:44771)
I20260806 22:37:20.170358 18122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:43313
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:20.198632 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40105
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:20.286582 18395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:20.286909 18395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:20.286959 18395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:20.288225 18395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:20.288311 18395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:20.288362 18395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:20.290010 18395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:20.290314 18395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:20.290582 18395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:37:20.296217 18395 server_base.cc:1034] running on GCE node
W20260806 22:37:20.296125 18404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:20.296125 18408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:20.296125 18403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:20.296665 18395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:20.296967 18395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:20.298127 18395 hybrid_clock.cc:648] HybridClock initialized: now 1786055840298114 us; error 32 us; skew 500 ppm
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:20.300997 18395 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:20.301456 18395 webserver.cc:460] Webserver started at http://127.16.235.66:40595/ using document root <none> and password file <none>
I20260806 22:37:20.301657 18395 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:20.301712 18395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:20.301819 18395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:20.303009 18395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "3dbd1e5cb51740ec8ba51ed61697f33f"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-3kk6"
server_key: "7dc5547b488d776ed797b2b590df0bcb"
server_key_iv: "961f9cb95460e50ee9f2264ea1afd358"
server_key_version: "encryptionkey@0"
I20260806 22:37:20.303352 18395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "3dbd1e5cb51740ec8ba51ed61697f33f"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-3kk6"
server_key: "7dc5547b488d776ed797b2b590df0bcb"
server_key_iv: "961f9cb95460e50ee9f2264ea1afd358"
server_key_version: "encryptionkey@0"
I20260806 22:37:20.304895 18395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:20.305763 18417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:20.305974 18395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:20.306098 18395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "3dbd1e5cb51740ec8ba51ed61697f33f"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-3kk6"
server_key: "7dc5547b488d776ed797b2b590df0bcb"
server_key_iv: "961f9cb95460e50ee9f2264ea1afd358"
server_key_version: "encryptionkey@0"
I20260806 22:37:20.306186 18395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:20.325577 18395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:20.328037 18395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:20.328240 18395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:20.328544 18395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:20.328905 18395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:20.328964 18395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:20.329003 18395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:20.329051 18395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:20.335811 18395 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:41283
I20260806 22:37:20.335856 18585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:41283 every 8 connection(s)
I20260806 22:37:20.336310 18395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:20.345486 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 18395
I20260806 22:37:20.345619 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:20.345800 17325 external_mini_cluster.cc:1428] Setting key 57ef7e5162a75d44fdbd989fbaf521e1
I20260806 22:37:20.347151 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.337149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:49633 (local address 127.16.235.126:40105)
0806 22:37:20.337216 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.337219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.337460 (+ 241us) server_negotiation.cc:407] Connection header received
0806 22:37:20.338492 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.338494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.338508 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.338578 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:20.339324 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.339859 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.340899 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.341116 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.341202 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.343603 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:20.343605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:20.343605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:20.343614 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:20.345175 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.345737 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.345738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.345738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.345809 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.346160 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.346161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.346162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.346356 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:20.346436 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.346901 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.347020 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:37:20.347282 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.337116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40105 (local address 127.16.235.66:49633)
0806 22:37:20.337306 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:20.337352 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:20.338367 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:20.338708 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:20.338712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:20.339005 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:20.339211 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.339216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.339967 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.339968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.340798 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.340800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.341211 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.341212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.341346 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.341920 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:20.341929 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:20.343479 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:20.345304 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:20.345314 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:20.345317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:20.345584 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:20.345884 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:20.345885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:20.345886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:20.346083 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:20.346453 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:20.346461 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:20.346692 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 22:37:20.346850 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 22:37:20.347003 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
I20260806 22:37:20.347777 18586 heartbeater.cc:347] Connected to a master server at 127.16.235.126:40105
I20260806 22:37:20.347913 18586 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:20.348114 18586 heartbeater.cc:511] Master 127.16.235.126:40105 requested a full tablet report, sending...
I20260806 22:37:20.348520 18122 ts_manager.cc:194] Registered new tserver with Master: 3dbd1e5cb51740ec8ba51ed61697f33f (127.16.235.66:41283)
I20260806 22:37:20.349080 18122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:49633
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:20.373947 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40105
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:20.461534 18592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:20.461818 18592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:20.461867 18592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:20.463131 18592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:20.463209 18592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:20.463259 18592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:20.464888 18592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:20.465198 18592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:20.465457 18592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:20.470973 18606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:20.471041 18592 server_base.cc:1034] running on GCE node
W20260806 22:37:20.470973 18602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:20.471006 18604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:20.471405 18592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:20.471796 18592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:20.472954 18592 hybrid_clock.cc:648] HybridClock initialized: now 1786055840472944 us; error 31 us; skew 500 ppm
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:20.475864 18592 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:20.476341 18592 webserver.cc:460] Webserver started at http://127.16.235.67:43923/ using document root <none> and password file <none>
I20260806 22:37:20.476564 18592 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:20.476608 18592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:20.476717 18592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:20.477825 18592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "b53deca0f5d848398de6da8a8991a6ce"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-3kk6"
server_key: "aa2bfabf10f54bbae1e134d5c2babf86"
server_key_iv: "5282e9c398c2a258856af1bea02377e2"
server_key_version: "encryptionkey@0"
I20260806 22:37:20.478149 18592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "b53deca0f5d848398de6da8a8991a6ce"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-3kk6"
server_key: "aa2bfabf10f54bbae1e134d5c2babf86"
server_key_iv: "5282e9c398c2a258856af1bea02377e2"
server_key_version: "encryptionkey@0"
I20260806 22:37:20.479728 18592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:37:20.480578 18615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:20.480778 18592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:20.480880 18592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "b53deca0f5d848398de6da8a8991a6ce"
format_stamp: "Formatted at 2026-08-06 22:37:20 on dist-test-slave-3kk6"
server_key: "aa2bfabf10f54bbae1e134d5c2babf86"
server_key_iv: "5282e9c398c2a258856af1bea02377e2"
server_key_version: "encryptionkey@0"
I20260806 22:37:20.480966 18592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:20.492852 18592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:20.495350 18592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:20.495514 18592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:20.495803 18592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:20.496150 18592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:20.496189 18592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:20.496214 18592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:20.496227 18592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:20.502789 18592 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:40207
I20260806 22:37:20.502853 18778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:40207 every 8 connection(s)
I20260806 22:37:20.503244 18592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:20.510445 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 18592
I20260806 22:37:20.510581 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:20.510799 17325 external_mini_cluster.cc:1428] Setting key 8001d0953adf6190cbcb1effe89095ac
I20260806 22:37:20.513947 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.504153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:59901 (local address 127.16.235.126:40105)
0806 22:37:20.504283 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.504286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.504505 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:37:20.505560 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.505562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.505576 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.505641 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:20.506357 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.506922 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.507980 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.508229 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.508311 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.511002 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:20.511004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:20.511004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:20.511012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:20.512623 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.513051 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.513052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.513052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.513110 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.513360 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.513360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.513361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.513488 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:20.513544 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.513800 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.513870 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:37:20.514132 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.504114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40105 (local address 127.16.235.67:59901)
0806 22:37:20.504346 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:20.504398 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:20.505433 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:20.505732 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:20.505736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:20.506032 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:20.506240 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.506245 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.507025 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.507026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.507878 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.507880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.508326 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.508327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.508466 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.509084 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:20.509092 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:20.510856 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:20.512702 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:20.512710 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:20.512712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:20.512975 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:20.513175 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:20.513175 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:20.513176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:20.513296 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:20.513548 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:20.513553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:20.513679 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:37:20.513770 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 22:37:20.513907 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":99,"threads_started":1}
I20260806 22:37:20.514638 18779 heartbeater.cc:347] Connected to a master server at 127.16.235.126:40105
I20260806 22:37:20.514832 18779 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:20.515025 18779 heartbeater.cc:511] Master 127.16.235.126:40105 requested a full tablet report, sending...
I20260806 22:37:20.515447 18122 ts_manager.cc:194] Registered new tserver with Master: b53deca0f5d848398de6da8a8991a6ce (127.16.235.67:40207)
I20260806 22:37:20.515839 18122 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:59901
I20260806 22:37:20.523066 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:37:20.531983 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.525251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.16.235.126:40105)
0806 22:37:20.525330 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.525334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.525348 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:37:20.525412 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.525413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.525423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.525485 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:20.526048 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.526550 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.527334 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.527527 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.527583 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.529212 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:20.529213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:20.529214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:20.529221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:20.530632 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.531158 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.531159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.531159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.531219 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.531398 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.531399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.531399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.531634 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:20.531732 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.531844 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.531914 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:37:20.535287 18122 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:20.541385 18122 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:20.550293 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.545569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36738 (local address 127.16.235.66:41283)
0806 22:37:20.545801 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.545807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.546003 (+ 196us) server_negotiation.cc:407] Connection header received
0806 22:37:20.546092 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.546095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.546113 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.546194 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:20.546499 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.547828 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.549123 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.549811 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.549890 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.549957 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.550173 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.550218 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":99,"threads_started":1}
I20260806 22:37:20.550268 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.545198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:44771 (local address 127.0.0.1:52980)
0806 22:37:20.545532 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:20.545545 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:20.545602 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:20.546509 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:20.546510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:20.546512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:20.546554 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.546555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.548139 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.548140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.548928 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.548931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.549922 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.549923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.550010 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.550015 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:37:20.550061 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:37:20.550122 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":121,"threads_started":1}
I20260806 22:37:20.550271 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.545004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:40207 (local address 127.0.0.1:42400)
0806 22:37:20.545131 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:20.545182 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:20.545275 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:20.546298 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:20.546302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:20.546304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:20.546398 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.546401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.547638 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.547639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.549223 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.549225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.549906 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.549907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.549992 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.550008 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:37:20.550115 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:37:20.550163 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":30,"thread_start_us":48,"threads_started":1}
I20260806 22:37:20.550261 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.545707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42400 (local address 127.16.235.67:40207)
0806 22:37:20.545954 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.545959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.545971 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:37:20.546022 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.546023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.546094 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.546490 (+ 396us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:20.546599 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.547510 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.549322 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.549812 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.549903 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.549963 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.550135 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.550172 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":140,"threads_started":1}
I20260806 22:37:20.550812 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.545340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.16.235.65:44771)
0806 22:37:20.546331 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.546335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.546345 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:37:20.546386 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.546388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.546403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.546632 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:20.546776 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.547902 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.549043 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.549811 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.550557 (+ 746us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.550630 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.550657 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.550724 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":858,"thread_start_us":79,"threads_started":1}
I20260806 22:37:20.550959 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.545429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:41283 (local address 127.0.0.1:36738)
0806 22:37:20.545858 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:20.545868 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:20.545928 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:20.546323 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:20.546324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:20.546327 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:20.546407 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.546409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.547940 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.547941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.549047 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.549050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.549987 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.549988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.550037 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.550041 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:20.550088 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:37:20.550122 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":67,"threads_started":1}
I20260806 22:37:20.551689 18301 tablet_service.cc:1467] Processing CreateTablet for tablet 6496869ae7954054b65cc78f6239f2e8 (DEFAULT_TABLE table=test-table [id=66487ff54df64de98b4e384a058b5362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:20.551697 18495 tablet_service.cc:1467] Processing CreateTablet for tablet 6496869ae7954054b65cc78f6239f2e8 (DEFAULT_TABLE table=test-table [id=66487ff54df64de98b4e384a058b5362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:20.552100 18301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6496869ae7954054b65cc78f6239f2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:20.552099 18495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6496869ae7954054b65cc78f6239f2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:20.555392 18680 tablet_service.cc:1467] Processing CreateTablet for tablet 6496869ae7954054b65cc78f6239f2e8 (DEFAULT_TABLE table=test-table [id=66487ff54df64de98b4e384a058b5362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:20.555770 18680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6496869ae7954054b65cc78f6239f2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:20.556119 18801 tablet_bootstrap.cc:492] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1: Bootstrap starting.
I20260806 22:37:20.556713 18801 tablet_bootstrap.cc:654] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:20.557402 18801 log.cc:824] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:20.557798 18803 tablet_bootstrap.cc:492] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f: Bootstrap starting.
I20260806 22:37:20.558188 18801 tablet_bootstrap.cc:492] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1: No bootstrap required, opened a new log
I20260806 22:37:20.558283 18801 ts_tablet_manager.cc:1397] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:37:20.558643 18803 tablet_bootstrap.cc:654] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:20.559351 18803 log.cc:824] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:20.560202 18803 tablet_bootstrap.cc:492] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f: No bootstrap required, opened a new log
I20260806 22:37:20.560266 18803 ts_tablet_manager.cc:1397] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:37:20.560284 18801 raft_consensus.cc:348] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } }
I20260806 22:37:20.560477 18801 raft_consensus.cc:374] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:20.560527 18801 raft_consensus.cc:729] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1cad0822ff045dba9206b682e97cac1, State: Initialized, Role: FOLLOWER
I20260806 22:37:20.560653 18801 consensus_queue.cc:260] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } }
I20260806 22:37:20.561021 18801 ts_tablet_manager.cc:1428] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:37:20.561088 18390 heartbeater.cc:503] Master 127.16.235.126:40105 was elected leader, sending a full tablet report...
I20260806 22:37:20.561233 18805 tablet_bootstrap.cc:492] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce: Bootstrap starting.
I20260806 22:37:20.562011 18805 tablet_bootstrap.cc:654] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:20.562726 18805 log.cc:824] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:20.562849 18803 raft_consensus.cc:348] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } }
I20260806 22:37:20.563045 18803 raft_consensus.cc:374] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:20.563097 18803 raft_consensus.cc:729] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dbd1e5cb51740ec8ba51ed61697f33f, State: Initialized, Role: FOLLOWER
I20260806 22:37:20.563220 18803 consensus_queue.cc:260] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } }
I20260806 22:37:20.563480 18805 tablet_bootstrap.cc:492] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce: No bootstrap required, opened a new log
I20260806 22:37:20.563536 18805 ts_tablet_manager.cc:1397] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:37:20.563526 18803 ts_tablet_manager.cc:1428] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:37:20.563526 18586 heartbeater.cc:503] Master 127.16.235.126:40105 was elected leader, sending a full tablet report...
I20260806 22:37:20.565122 18805 raft_consensus.cc:348] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } }
I20260806 22:37:20.565265 18805 raft_consensus.cc:374] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:20.565289 18805 raft_consensus.cc:729] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53deca0f5d848398de6da8a8991a6ce, State: Initialized, Role: FOLLOWER
I20260806 22:37:20.565425 18805 consensus_queue.cc:260] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } }
I20260806 22:37:20.565689 18805 ts_tablet_manager.cc:1428] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:37:20.565826 18779 heartbeater.cc:503] Master 127.16.235.126:40105 was elected leader, sending a full tablet report...
W20260806 22:37:20.587004 18587 tablet.cc:2369] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:20.657214 18391 tablet.cc:2369] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:20.753919 18780 tablet.cc:2369] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:20.877657 18810 raft_consensus.cc:482] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:20.877818 18810 raft_consensus.cc:504] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } }
I20260806 22:37:20.878449 18810 leader_election.cc:288] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1cad0822ff045dba9206b682e97cac1 (127.16.235.65:44771), 3dbd1e5cb51740ec8ba51ed61697f33f (127.16.235.66:41283)
I20260806 22:37:20.881635 18781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.878775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:44771 (local address 127.16.235.67:40943)
0806 22:37:20.878870 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:20.878885 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:20.878983 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:20.879147 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:20.879148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:20.879150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:20.879302 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.879305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.880011 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.880012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.880804 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.880806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.881368 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.881370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.881451 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.881454 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:20.881506 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:37:20.881539 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":15}
I20260806 22:37:20.881646 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.878798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:40943 (local address 127.16.235.65:44771)
0806 22:37:20.878875 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.878878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.878962 (+ 84us) server_negotiation.cc:407] Connection header received
0806 22:37:20.879049 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.879050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.879061 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.879127 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:20.879387 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.879925 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.880904 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.881283 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.881348 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.881356 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.881531 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.881577 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:37:20.881712 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.879010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:52323 (local address 127.16.235.66:41283)
0806 22:37:20.879056 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.879059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.879182 (+ 123us) server_negotiation.cc:407] Connection header received
0806 22:37:20.879256 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.879257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.879270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.879343 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:20.879517 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.879984 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.880966 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.881314 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.881406 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.881414 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.881597 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.881634 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:37:20.881771 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.878930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:41283 (local address 127.16.235.67:52323)
0806 22:37:20.879113 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:20.879123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:20.879175 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:20.879373 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:20.879374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:20.879376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:20.879448 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.879449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.880079 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.880080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.880872 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:20.880875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.881427 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.881428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:20.881512 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.881517 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:37:20.881578 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:37:20.881616 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260806 22:37:20.882099 18326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6496869ae7954054b65cc78f6239f2e8" candidate_uuid: "b53deca0f5d848398de6da8a8991a6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1cad0822ff045dba9206b682e97cac1" is_pre_election: true
I20260806 22:37:20.882320 18326 raft_consensus.cc:2454] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b53deca0f5d848398de6da8a8991a6ce in term 0.
I20260806 22:37:20.882607 18620 leader_election.cc:302] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b53deca0f5d848398de6da8a8991a6ce, c1cad0822ff045dba9206b682e97cac1; no voters:
I20260806 22:37:20.882761 18810 raft_consensus.cc:2781] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:20.882860 18810 raft_consensus.cc:482] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:20.882900 18810 raft_consensus.cc:3037] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:20.882099 18523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6496869ae7954054b65cc78f6239f2e8" candidate_uuid: "b53deca0f5d848398de6da8a8991a6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" is_pre_election: true
I20260806 22:37:20.883764 18810 raft_consensus.cc:504] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } }
I20260806 22:37:20.883929 18810 leader_election.cc:288] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [CANDIDATE]: Term 1 election: Requested vote from peers c1cad0822ff045dba9206b682e97cac1 (127.16.235.65:44771), 3dbd1e5cb51740ec8ba51ed61697f33f (127.16.235.66:41283)
I20260806 22:37:20.883899 18523 raft_consensus.cc:2454] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b53deca0f5d848398de6da8a8991a6ce in term 0.
I20260806 22:37:20.884135 18326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6496869ae7954054b65cc78f6239f2e8" candidate_uuid: "b53deca0f5d848398de6da8a8991a6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1cad0822ff045dba9206b682e97cac1"
I20260806 22:37:20.884195 18523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6496869ae7954054b65cc78f6239f2e8" candidate_uuid: "b53deca0f5d848398de6da8a8991a6ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f"
I20260806 22:37:20.884219 18326 raft_consensus.cc:3037] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:20.884289 18523 raft_consensus.cc:3037] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:20.885054 18523 raft_consensus.cc:2454] T 6496869ae7954054b65cc78f6239f2e8 P 3dbd1e5cb51740ec8ba51ed61697f33f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b53deca0f5d848398de6da8a8991a6ce in term 1.
I20260806 22:37:20.885064 18326 raft_consensus.cc:2454] T 6496869ae7954054b65cc78f6239f2e8 P c1cad0822ff045dba9206b682e97cac1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b53deca0f5d848398de6da8a8991a6ce in term 1.
I20260806 22:37:20.885234 18620 leader_election.cc:302] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dbd1e5cb51740ec8ba51ed61697f33f, b53deca0f5d848398de6da8a8991a6ce; no voters:
I20260806 22:37:20.885366 18810 raft_consensus.cc:2781] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:20.885583 18810 raft_consensus.cc:686] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [term 1 LEADER]: Becoming Leader. State: Replica: b53deca0f5d848398de6da8a8991a6ce, State: Running, Role: LEADER
I20260806 22:37:20.885689 18810 consensus_queue.cc:237] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } }
I20260806 22:37:20.886607 18122 catalog_manager.cc:5495] T 6496869ae7954054b65cc78f6239f2e8 P b53deca0f5d848398de6da8a8991a6ce reported cstate change: term changed from 0 to 1, leader changed from <none> to b53deca0f5d848398de6da8a8991a6ce (127.16.235.67). New cstate: current_term: 1 leader_uuid: "b53deca0f5d848398de6da8a8991a6ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53deca0f5d848398de6da8a8991a6ce" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 40207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1cad0822ff045dba9206b682e97cac1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dbd1e5cb51740ec8ba51ed61697f33f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41283 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:20.909644 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.901407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52984 (local address 127.16.235.65:44771)
0806 22:37:20.901494 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.901496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.901520 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:37:20.901594 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.901595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.901606 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.901661 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:20.902223 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.902779 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.903546 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.903732 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.903796 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.906452 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:20.906466 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:20.906466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:20.906475 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:20.908344 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.908711 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.908715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.908716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.908796 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.908993 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.908994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.908994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.909190 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:20.909362 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.909407 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.909483 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:20.935635 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.927615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.16.235.65:44771)
0806 22:37:20.927694 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.927696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.927754 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:37:20.927807 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.927809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.927825 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.927884 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:20.928452 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.928936 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.929863 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.930089 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.930152 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.932914 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:20.932915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:20.932916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:20.932923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:20.934300 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.934756 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.934758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.934758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.934830 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.935031 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.935032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.935032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.935135 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:20.935212 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.935420 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.935498 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:37:20 dist-test-slave-3kk6 krb5kdc[18057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:20.963455 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:20.955549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53004 (local address 127.16.235.65:44771)
0806 22:37:20.955635 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:20.955637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:20.955724 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:37:20.955843 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:20.955844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:20.955863 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:20.955931 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:20.956562 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.957070 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.957842 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:20.958044 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:20.958127 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:20.960570 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:20.960572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:20.960572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:20.960580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:20.962098 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.962523 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.962525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.962525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.962636 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:20.962890 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:20.962892 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:20.962892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:20.963007 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:20.963121 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:20.963263 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:37:20.963349 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 22:37:20.964243 18268 tablet_service.cc:2908] Rejecting scan request for tablet 6496869ae7954054b65cc78f6239f2e8: Uninitialized: clean time has not yet been initialized
I20260806 22:37:20.965425 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 18207
I20260806 22:37:20.975898 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 18395
I20260806 22:37:21.000653 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 18592
I20260806 22:37:21.010521 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 18076
2026-08-06T22:37:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2560 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:21 dist-test-slave-3kk6 krb5kdc[18834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:21 dist-test-slave-3kk6 krb5kdc[18834](info): set up 2 sockets
Aug 06 22:37:21 dist-test-slave-3kk6 krb5kdc[18834](info): commencing operation
krb5kdc: starting...
W20260806 22:37:22.245059 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:22 dist-test-slave-3kk6 krb5kdc[18834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:22.377579 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40289
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:40289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:22.464498 18849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:22.464761 18849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:22.464813 18849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:22.466042 18849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:22.466115 18849 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:37:22.466137 18849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:22.466159 18849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:22.466192 18849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:22.466215 18849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:22.467820 18849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:40289
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:22.468142 18849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:22.468400 18849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:22.473760 18858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:22.474010 18849 server_base.cc:1034] running on GCE node
W20260806 22:37:22.473766 18857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:22.473766 18860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:22.474361 18849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:22.474736 18849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:22.475912 18849 hybrid_clock.cc:648] HybridClock initialized: now 1786055842475892 us; error 43 us; skew 500 ppm
Aug 06 22:37:22 dist-test-slave-3kk6 krb5kdc[18834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:22.478802 18849 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:22.479272 18849 webserver.cc:460] Webserver started at http://127.16.235.126:44123/ using document root <none> and password file <none>
I20260806 22:37:22.479524 18849 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:22.479575 18849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:22.479720 18849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:22.480885 18849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "800924947cdd471d87c0ecb4a3b6f401"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-3kk6"
server_key: "25e103602cc01881688bfb0c457dce50"
server_key_iv: "f2a6c85e26e8836cc215c7842302bfdd"
server_key_version: "encryptionkey@0"
I20260806 22:37:22.481289 18849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "800924947cdd471d87c0ecb4a3b6f401"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-3kk6"
server_key: "25e103602cc01881688bfb0c457dce50"
server_key_iv: "f2a6c85e26e8836cc215c7842302bfdd"
server_key_version: "encryptionkey@0"
I20260806 22:37:22.482841 18849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:37:22.483750 18867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:22.483973 18849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:37:22.484079 18849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "800924947cdd471d87c0ecb4a3b6f401"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-3kk6"
server_key: "25e103602cc01881688bfb0c457dce50"
server_key_iv: "f2a6c85e26e8836cc215c7842302bfdd"
server_key_version: "encryptionkey@0"
I20260806 22:37:22.484161 18849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:22.493007 18849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:22.495433 18849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:22.495605 18849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:22.500996 18849 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:40289
I20260806 22:37:22.501089 18959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:40289 every 8 connection(s)
I20260806 22:37:22.501520 18849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:22.502338 18961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:22.504076 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 18849
I20260806 22:37:22.504189 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:22.504375 17325 external_mini_cluster.cc:1428] Setting key 0fcb294a06ea32ab42a1d1266f57e47a
I20260806 22:37:22.505435 18961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401: Bootstrap starting.
I20260806 22:37:22.506356 18961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:22.506824 18961 log.cc:824] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:22.507759 18961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401: No bootstrap required, opened a new log
I20260806 22:37:22.510856 18961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "800924947cdd471d87c0ecb4a3b6f401" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40289 } }
I20260806 22:37:22.511063 18961 raft_consensus.cc:374] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:22.511091 18961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800924947cdd471d87c0ecb4a3b6f401, State: Initialized, Role: FOLLOWER
I20260806 22:37:22.511263 18961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "800924947cdd471d87c0ecb4a3b6f401" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40289 } }
I20260806 22:37:22.511353 18961 raft_consensus.cc:388] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:22.511379 18961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:22.511457 18961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:22.512271 18961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "800924947cdd471d87c0ecb4a3b6f401" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40289 } }
I20260806 22:37:22.512410 18961 leader_election.cc:302] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 800924947cdd471d87c0ecb4a3b6f401; no voters:
Aug 06 22:37:22 dist-test-slave-3kk6 krb5kdc[18834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:22.512665 18961 leader_election.cc:288] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:22.512702 18967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:22.512920 18967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [term 1 LEADER]: Becoming Leader. State: Replica: 800924947cdd471d87c0ecb4a3b6f401, State: Running, Role: LEADER
I20260806 22:37:22.513090 18967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "800924947cdd471d87c0ecb4a3b6f401" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40289 } }
I20260806 22:37:22.513091 18961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:22.513654 18970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "800924947cdd471d87c0ecb4a3b6f401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "800924947cdd471d87c0ecb4a3b6f401" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40289 } } }
I20260806 22:37:22.513670 18969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 800924947cdd471d87c0ecb4a3b6f401. Latest consensus state: current_term: 1 leader_uuid: "800924947cdd471d87c0ecb4a3b6f401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "800924947cdd471d87c0ecb4a3b6f401" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40289 } } }
I20260806 22:37:22.513888 18969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:22.514201 18977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:22.513836 18970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:22.514926 18977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:22.516917 18977 catalog_manager.cc:1347] Generated new cluster ID: 4481d658836940f98cb9fd53d49082f0
I20260806 22:37:22.516978 18977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:22.517634 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.505014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35194 (local address 127.16.235.126:40289)
0806 22:37:22.505229 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:22.505240 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:22.505276 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:37:22.505885 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:22.505896 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:22.505978 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:22.506173 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:22.506915 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.508373 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.509330 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.510046 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.510179 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.513249 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:22.513255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:22.513256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:22.513273 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:22.515764 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.516283 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.516287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.516288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.516413 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.516664 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.516665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.516665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.517027 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:22.517169 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:22.517304 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:37:22.517457 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260806 22:37:22.539575 18977 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:22.540232 18977 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:22.549141 18977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401: Generated new TSK 0
I20260806 22:37:22.549420 18977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:22.587121 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40289
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:22.679351 18997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:22.679608 18997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:22.679636 18997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:22.680851 18997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:22.680907 18997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:22.680929 18997 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:37:22.680948 18997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:22.682482 18997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.235.126:40289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:22.682787 18997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:22.683017 18997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:37:22.688778 18997 server_base.cc:1034] running on GCE node
W20260806 22:37:22.688740 19009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:22.688637 19011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:22.688637 19014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:22.689190 18997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:22.689489 18997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:22.690713 18997 hybrid_clock.cc:648] HybridClock initialized: now 1786055842690703 us; error 40 us; skew 500 ppm
Aug 06 22:37:22 dist-test-slave-3kk6 krb5kdc[18834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:22.693547 18997 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:22.694083 18997 webserver.cc:460] Webserver started at http://127.16.235.65:37269/ using document root <none> and password file <none>
I20260806 22:37:22.694386 18997 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:22.694463 18997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:22.694638 18997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:22.695861 18997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "0fd9c07544bc474b91162e1cd6604f2b"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-3kk6"
server_key: "98220d3709b6032829a0974d1a70c5fa"
server_key_iv: "fbd2453d55da96de6707e114cf4fd0b8"
server_key_version: "encryptionkey@0"
I20260806 22:37:22.696277 18997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "0fd9c07544bc474b91162e1cd6604f2b"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-3kk6"
server_key: "98220d3709b6032829a0974d1a70c5fa"
server_key_iv: "fbd2453d55da96de6707e114cf4fd0b8"
server_key_version: "encryptionkey@0"
I20260806 22:37:22.697826 18997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:22.698729 19022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:22.698961 18997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:22.699081 18997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "0fd9c07544bc474b91162e1cd6604f2b"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-3kk6"
server_key: "98220d3709b6032829a0974d1a70c5fa"
server_key_iv: "fbd2453d55da96de6707e114cf4fd0b8"
server_key_version: "encryptionkey@0"
I20260806 22:37:22.699168 18997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:22.716521 18997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:22.719269 18997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:22.719476 18997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:22.720106 18997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:22.720172 18997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:22.720209 18997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:22.720238 18997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:37:22 dist-test-slave-3kk6 krb5kdc[18834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:22.728633 18997 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:35027
I20260806 22:37:22.729034 18997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:22.729034 19189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:35027 every 8 connection(s)
I20260806 22:37:22.731467 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.720975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:47185 (local address 127.16.235.126:40289)
0806 22:37:22.721033 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:22.721035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:22.721305 (+ 270us) server_negotiation.cc:407] Connection header received
0806 22:37:22.722083 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:22.722086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:22.722105 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:22.722196 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:22.723176 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.723821 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.724925 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.725158 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.725252 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.728329 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:22.728331 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:22.728331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:22.728340 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:22.730088 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.730507 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.730508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.730508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.730571 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.730894 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.730895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.730895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.731022 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:22.731082 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:22.731314 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 22:37:22.731398 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:37:22.731681 19034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.720977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40289 (local address 127.16.235.65:47185)
0806 22:37:22.721154 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:22.721197 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:22.721909 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:22.722320 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:22.722326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:22.722802 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:22.723048 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.723053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.723943 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.723944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.724813 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.724815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.725284 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.725285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.725449 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.726148 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:22.726163 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:22.728203 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:22.730180 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.730186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.730188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.730428 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.730636 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.730637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.730637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.730827 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.731085 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:22.731090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:22.731215 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 22:37:22.731295 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:37:22.731429 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260806 22:37:22.732257 19191 heartbeater.cc:347] Connected to a master server at 127.16.235.126:40289
I20260806 22:37:22.732384 19191 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:22.732601 19191 heartbeater.cc:511] Master 127.16.235.126:40289 requested a full tablet report, sending...
I20260806 22:37:22.733184 18888 ts_manager.cc:194] Registered new tserver with Master: 0fd9c07544bc474b91162e1cd6604f2b (127.16.235.65:35027)
I20260806 22:37:22.734053 18888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:47185
I20260806 22:37:22.734053 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 18997
I20260806 22:37:22.734181 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:22.734380 17325 external_mini_cluster.cc:1428] Setting key b208271d239c2902038abd67305aefd0
I20260806 22:37:22.740154 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.733519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:37755 (local address 127.16.235.126:40289)
0806 22:37:22.733564 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:22.733566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:22.733702 (+ 136us) server_negotiation.cc:407] Connection header received
0806 22:37:22.733731 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:22.733732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:22.733743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:22.733812 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:22.734287 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.734980 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.735825 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.736037 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.736111 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.737272 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:22.737274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:22.737274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:22.737281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:22.738860 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.739315 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.739316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.739316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.739374 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.739584 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.739584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.739584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.739695 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:22.739747 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:22.739940 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:37:22.740036 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:37:22.740237 19202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.733454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40289 (local address 127.16.235.65:37755)
0806 22:37:22.733631 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:22.733641 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:22.733694 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:22.733814 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:22.733815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:22.734121 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:22.734202 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.734204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.735073 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.735074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.735735 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.735738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.736111 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.736111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.736196 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.736767 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:22.736772 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:22.737182 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:22.738947 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.738949 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.738949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.739209 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.739439 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.739440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.739440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.739519 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.739768 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:22.739769 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:22.739835 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 22:37:22.739922 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:37:22.740058 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:22.765596 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40289
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:22.852442 19205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:22.852697 19205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:22.852726 19205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:22.854053 19205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:22.854112 19205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:22.854136 19205 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:37:22.854174 19205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:22.855775 19205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.235.126:40289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:22.856058 19205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:22.856324 19205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:22.862020 19221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:22.862054 19215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:22.862344 19205 server_base.cc:1034] running on GCE node
W20260806 22:37:22.862072 19213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:22.862805 19205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:22.863070 19205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:22.864231 19205 hybrid_clock.cc:648] HybridClock initialized: now 1786055842864218 us; error 34 us; skew 500 ppm
Aug 06 22:37:22 dist-test-slave-3kk6 krb5kdc[18834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:22.867246 19205 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:22.867835 19205 webserver.cc:460] Webserver started at http://127.16.235.66:43891/ using document root <none> and password file <none>
I20260806 22:37:22.868098 19205 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:22.868173 19205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:22.868338 19205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:22.869496 19205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "1c4bd0f096b04c0690132264a36808b4"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-3kk6"
server_key: "75889ae88731b1a89eedb4f8fe4a6e77"
server_key_iv: "33d84383585a48b9738440bd7c961f0c"
server_key_version: "encryptionkey@0"
I20260806 22:37:22.869923 19205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "1c4bd0f096b04c0690132264a36808b4"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-3kk6"
server_key: "75889ae88731b1a89eedb4f8fe4a6e77"
server_key_iv: "33d84383585a48b9738440bd7c961f0c"
server_key_version: "encryptionkey@0"
I20260806 22:37:22.871640 19205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260806 22:37:22.872510 19232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:22.872697 19205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:22.872856 19205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "1c4bd0f096b04c0690132264a36808b4"
format_stamp: "Formatted at 2026-08-06 22:37:22 on dist-test-slave-3kk6"
server_key: "75889ae88731b1a89eedb4f8fe4a6e77"
server_key_iv: "33d84383585a48b9738440bd7c961f0c"
server_key_version: "encryptionkey@0"
I20260806 22:37:22.872943 19205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:22.888967 19205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:22.891505 19205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:22.891716 19205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:22.892700 19205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:22.892755 19205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:22.892786 19205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:22.892800 19205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:37:22 dist-test-slave-3kk6 krb5kdc[18834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:22.900873 19205 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:33371
I20260806 22:37:22.901016 19388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:33371 every 8 connection(s)
I20260806 22:37:22.901434 19205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:22.902065 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 19205
I20260806 22:37:22.902171 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:22.902417 17325 external_mini_cluster.cc:1428] Setting key 5fa2b0c2ad1b9b82b4c79ed2d460445d
I20260806 22:37:22.904239 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.893174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:57183 (local address 127.16.235.126:40289)
0806 22:37:22.893257 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:22.893260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:22.893536 (+ 276us) server_negotiation.cc:407] Connection header received
0806 22:37:22.894365 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:22.894367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:22.894381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:22.894452 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:22.895272 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.895819 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.896890 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.897093 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.897203 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.900455 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:22.900456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:22.900457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:22.900464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:22.902410 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.903024 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.903026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.903026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.903110 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.903473 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.903474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.903475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.903639 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:22.903713 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:22.903998 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:37:22.904097 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:37:22.904258 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.893132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40289 (local address 127.16.235.66:57183)
0806 22:37:22.893366 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:22.893423 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:22.894196 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:22.894551 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:22.894556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:22.894940 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:22.895162 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.895168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.895929 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.895930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.896773 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.896776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.897216 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.897217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.897362 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.897982 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:22.897991 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:22.900332 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:22.902536 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.902546 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.902549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.902941 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.903190 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.903191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.903191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.903375 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.903709 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:22.903714 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:22.903853 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 22:37:22.903936 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:37:22.904073 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":111,"threads_started":1}
I20260806 22:37:22.905211 19389 heartbeater.cc:347] Connected to a master server at 127.16.235.126:40289
I20260806 22:37:22.905321 19389 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:22.905555 19389 heartbeater.cc:511] Master 127.16.235.126:40289 requested a full tablet report, sending...
I20260806 22:37:22.905956 18894 ts_manager.cc:194] Registered new tserver with Master: 1c4bd0f096b04c0690132264a36808b4 (127.16.235.66:33371)
I20260806 22:37:22.906502 18894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:57183
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
I20260806 22:37:22.912982 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.906000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40289 (local address 127.16.235.66:55511)
0806 22:37:22.906146 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:22.906157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:22.906219 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:22.906394 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:22.906395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:22.906739 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:22.906822 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.906825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.907684 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.907685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.908279 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:22.908281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.908788 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.908789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:22.908862 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.909392 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:22.909397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:22.909715 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:22.911777 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.911778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.911778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.911971 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.912202 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:22.912202 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:22.912203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:22.912268 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:22.912642 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:22.912643 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:22.912688 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:37:22.912742 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:37:22.912835 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260806 22:37:22.913143 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:22.906052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:55511 (local address 127.16.235.126:40289)
0806 22:37:22.906087 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:22.906089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:22.906220 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:37:22.906279 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:22.906280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:22.906290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:22.906347 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:22.906891 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.907581 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.908368 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:22.908691 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:22.908773 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:22.909813 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:22.909815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:22.909815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:22.909822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:22.911648 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.912059 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.912060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.912061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.912126 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:22.912339 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:22.912340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:22.912340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:22.912570 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:22.912649 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:22.912797 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 22:37:22.912964 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:22.934870 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40289
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:23.041740 19407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:23.042282 19407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:23.042339 19407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:23.044718 19407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:23.044938 19407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:23.044987 19407 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:37:23.045042 19407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:23.048429 19407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.235.126:40289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:23.049046 19407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:23.049721 19407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:23.061823 19415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:23.061977 19416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:23.061833 19419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:23.063060 19407 server_base.cc:1034] running on GCE node
I20260806 22:37:23.063686 19407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:23.064441 19407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:23.065752 19407 hybrid_clock.cc:648] HybridClock initialized: now 1786055843065664 us; error 115 us; skew 500 ppm
Aug 06 22:37:23 dist-test-slave-3kk6 krb5kdc[18834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055843, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:23.072305 19407 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:23.073688 19407 webserver.cc:460] Webserver started at http://127.16.235.67:46665/ using document root <none> and password file <none>
I20260806 22:37:23.074352 19407 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:23.074563 19407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:23.074949 19407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:23.077464 19407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "25e5f01417ca45d5b106c1ecc4e78cd8"
format_stamp: "Formatted at 2026-08-06 22:37:23 on dist-test-slave-3kk6"
server_key: "1cb992b2ddece4a2c0274f5a803e1a65"
server_key_iv: "e493a8f0e94603fab322e6c87498203a"
server_key_version: "encryptionkey@0"
I20260806 22:37:23.078341 19407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "25e5f01417ca45d5b106c1ecc4e78cd8"
format_stamp: "Formatted at 2026-08-06 22:37:23 on dist-test-slave-3kk6"
server_key: "1cb992b2ddece4a2c0274f5a803e1a65"
server_key_iv: "e493a8f0e94603fab322e6c87498203a"
server_key_version: "encryptionkey@0"
I20260806 22:37:23.081513 19407 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260806 22:37:23.083345 19429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:23.083772 19407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:37:23.083984 19407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "25e5f01417ca45d5b106c1ecc4e78cd8"
format_stamp: "Formatted at 2026-08-06 22:37:23 on dist-test-slave-3kk6"
server_key: "1cb992b2ddece4a2c0274f5a803e1a65"
server_key_iv: "e493a8f0e94603fab322e6c87498203a"
server_key_version: "encryptionkey@0"
I20260806 22:37:23.084131 19407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:23.104419 19407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:23.108601 19407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:23.108903 19407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:23.109861 19407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:23.109948 19407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:23.110003 19407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:23.110034 19407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:37:23 dist-test-slave-3kk6 krb5kdc[18834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055843, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:23.124768 19407 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:36833
I20260806 22:37:23.124981 19576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:36833 every 8 connection(s)
I20260806 22:37:23.125531 19407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:23.129542 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.110711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:39109 (local address 127.16.235.126:40289)
0806 22:37:23.110792 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.110795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.111334 (+ 539us) server_negotiation.cc:407] Connection header received
0806 22:37:23.112584 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.112585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.112606 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.112785 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.114755 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.115685 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.118499 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.118961 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.119135 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.123408 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.123411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.123412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.123451 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.126890 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.127673 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.127675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.127675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.127779 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.128267 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.128274 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.128274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.128487 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.128622 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.129230 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.129378 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:37:23.129914 19440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.110722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40289 (local address 127.16.235.67:39109)
0806 22:37:23.111026 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.111134 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.112352 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.112981 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.112987 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.113826 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:23.114462 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.114472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.115892 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.115894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.118329 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.118334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.119198 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.119205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.119431 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.120491 (+ 1060us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:23.120508 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:23.123167 (+ 2659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:23.127071 (+ 3904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:23.127084 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:23.127087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:23.127518 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:23.127925 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:23.127926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:23.127926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:23.128119 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:23.128628 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:23.128650 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:23.128954 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.129178 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.129510 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":149,"threads_started":1}
I20260806 22:37:23.130771 19578 heartbeater.cc:347] Connected to a master server at 127.16.235.126:40289
I20260806 22:37:23.130983 19578 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:23.131304 19578 heartbeater.cc:511] Master 127.16.235.126:40289 requested a full tablet report, sending...
I20260806 22:37:23.132052 18894 ts_manager.cc:194] Registered new tserver with Master: 25e5f01417ca45d5b106c1ecc4e78cd8 (127.16.235.67:36833)
I20260806 22:37:23.132627 18894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:39109
I20260806 22:37:23.133561 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 19407
I20260806 22:37:23.133736 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:23.134028 17325 external_mini_cluster.cc:1428] Setting key 3693b898f7c6ce88ea0d6570aa14304f
I20260806 22:37:23.135385 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:37:23.142845 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.133161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:39071 (local address 127.16.235.126:40289)
0806 22:37:23.133225 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.133227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.133237 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:37:23.133348 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.133349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.133361 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.133458 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.134200 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.135228 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.136279 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.136562 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.136681 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.138276 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.138277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.138278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.138286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.140734 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.141407 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.141408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.141409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.141523 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.141894 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.141895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.141895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.142082 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.142215 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.142487 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.142696 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:37:23.142863 19589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.132842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40289 (local address 127.16.235.67:39071)
0806 22:37:23.133128 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.133141 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.133233 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.133471 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.133472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.133875 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:23.134009 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.134012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.135385 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.135386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.136116 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.136118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.136682 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.136684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.136826 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.137605 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:23.137610 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:23.138119 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:23.140940 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:23.140942 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:23.140943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:23.141269 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:23.141625 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:23.141626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:23.141626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:23.141770 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:23.142200 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:23.142201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:23.142300 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.142430 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.142711 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":154,"threads_started":1}
Aug 06 22:37:23 dist-test-slave-3kk6 krb5kdc[18834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:37:23 dist-test-slave-3kk6 krb5kdc[18834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:23.177201 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.164891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35202 (local address 127.16.235.126:40289)
0806 22:37:23.165097 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.165106 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.165117 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:23.165167 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.165168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.165185 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.165272 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.166374 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.167124 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.168165 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.168437 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.168530 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.172626 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.172629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.172629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.172638 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.175155 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.175851 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.175852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.175853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.175965 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.176302 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.176303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.176303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.176553 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.176663 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.176940 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.177067 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260806 22:37:23.180333 18894 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:23.183251 18894 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:23.196208 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.187743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:35027 (local address 127.0.0.1:52934)
0806 22:37:23.188750 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.188763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.188824 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.189465 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.189466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.189473 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:23.189565 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.189567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.191154 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.191155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.193695 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.193697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.195351 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.195353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.195515 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.195547 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.195806 (+ 259us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.195945 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":56,"thread_start_us":155,"threads_started":1}
I20260806 22:37:23.196872 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.188419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60640 (local address 127.16.235.66:33371)
0806 22:37:23.189045 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.189056 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.189074 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:37:23.189146 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.189152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.189182 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.189294 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:23.189725 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.191333 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.193651 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.195131 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.195285 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.195412 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.195841 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.196654 (+ 813us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":170,"threads_started":1}
I20260806 22:37:23.197242 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.188445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54928 (local address 127.16.235.67:36833)
0806 22:37:23.188788 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.188798 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.188968 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:37:23.189064 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.189073 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.189113 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.189273 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:23.189525 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.191304 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.194880 (+ 3576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.196403 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.196559 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.196661 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.197012 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.197073 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":170,"threads_started":1}
I20260806 22:37:23.197348 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.187739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:33371 (local address 127.0.0.1:60640)
0806 22:37:23.188258 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.188343 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.188499 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.189393 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.189394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.189396 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:23.189568 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.189580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.191549 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.191550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.193440 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.193443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.195479 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.195480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.195579 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.195587 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.195813 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.196946 (+ 1133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":1002,"thread_start_us":173,"threads_started":1}
I20260806 22:37:23.197403 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.188206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:36833 (local address 127.0.0.1:54928)
0806 22:37:23.188853 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.188870 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.188939 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.189285 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.189288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.189290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:23.189408 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.189412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.191469 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.191470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.193692 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.193696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.196705 (+ 3009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.196706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.196771 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.196779 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.196984 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.197061 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":99,"thread_start_us":123,"threads_started":1}
I20260806 22:37:23.196303 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.188038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52934 (local address 127.16.235.65:35027)
0806 22:37:23.189097 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.189101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.189118 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:37:23.189178 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.189180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.189239 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.189372 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:23.189687 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.190990 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.193835 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.195223 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.195322 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.195412 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.195926 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.196017 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1013,"thread_start_us":169,"threads_started":1}
I20260806 22:37:23.198982 19496 tablet_service.cc:1467] Processing CreateTablet for tablet b776cb33d3694f38b5d12588053d0fbc (DEFAULT_TABLE table=test-table [id=fc58f618318a4145aae910a7d9669d2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:23.198992 19302 tablet_service.cc:1467] Processing CreateTablet for tablet b776cb33d3694f38b5d12588053d0fbc (DEFAULT_TABLE table=test-table [id=fc58f618318a4145aae910a7d9669d2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:23.199186 19093 tablet_service.cc:1467] Processing CreateTablet for tablet b776cb33d3694f38b5d12588053d0fbc (DEFAULT_TABLE table=test-table [id=fc58f618318a4145aae910a7d9669d2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:23.199896 19302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b776cb33d3694f38b5d12588053d0fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:23.199896 19496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b776cb33d3694f38b5d12588053d0fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:23.199939 19093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b776cb33d3694f38b5d12588053d0fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:23.205765 19614 tablet_bootstrap.cc:492] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: Bootstrap starting.
I20260806 22:37:23.207525 19615 tablet_bootstrap.cc:492] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: Bootstrap starting.
I20260806 22:37:23.207569 19616 tablet_bootstrap.cc:492] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: Bootstrap starting.
I20260806 22:37:23.208550 19614 tablet_bootstrap.cc:654] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:23.208869 19616 tablet_bootstrap.cc:654] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:23.209343 19614 log.cc:824] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:23.209486 19616 log.cc:824] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:23.209924 19615 tablet_bootstrap.cc:654] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:23.210592 19615 log.cc:824] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:23.211337 19614 tablet_bootstrap.cc:492] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: No bootstrap required, opened a new log
I20260806 22:37:23.211544 19614 ts_tablet_manager.cc:1397] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 22:37:23.212339 19616 tablet_bootstrap.cc:492] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: No bootstrap required, opened a new log
I20260806 22:37:23.212471 19616 ts_tablet_manager.cc:1397] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 22:37:23.212577 19615 tablet_bootstrap.cc:492] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: No bootstrap required, opened a new log
I20260806 22:37:23.212707 19615 ts_tablet_manager.cc:1397] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 22:37:23.215757 19615 raft_consensus.cc:348] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.216055 19615 raft_consensus.cc:374] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:23.216136 19615 raft_consensus.cc:729] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e5f01417ca45d5b106c1ecc4e78cd8, State: Initialized, Role: FOLLOWER
I20260806 22:37:23.216382 19615 consensus_queue.cc:260] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.216177 19614 raft_consensus.cc:348] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.216560 19614 raft_consensus.cc:374] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:23.216657 19614 raft_consensus.cc:729] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd9c07544bc474b91162e1cd6604f2b, State: Initialized, Role: FOLLOWER
I20260806 22:37:23.216971 19614 consensus_queue.cc:260] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.216777 19616 raft_consensus.cc:348] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.217048 19615 ts_tablet_manager.cc:1428] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260806 22:37:23.217119 19616 raft_consensus.cc:374] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:23.217202 19616 raft_consensus.cc:729] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4bd0f096b04c0690132264a36808b4, State: Initialized, Role: FOLLOWER
I20260806 22:37:23.217281 19578 heartbeater.cc:503] Master 127.16.235.126:40289 was elected leader, sending a full tablet report...
I20260806 22:37:23.217541 19614 ts_tablet_manager.cc:1428] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260806 22:37:23.217521 19616 consensus_queue.cc:260] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.217761 19191 heartbeater.cc:503] Master 127.16.235.126:40289 was elected leader, sending a full tablet report...
I20260806 22:37:23.218116 19616 ts_tablet_manager.cc:1428] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 22:37:23.218225 19389 heartbeater.cc:503] Master 127.16.235.126:40289 was elected leader, sending a full tablet report...
W20260806 22:37:23.230173 19192 tablet.cc:2369] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:23.255349 19623 raft_consensus.cc:482] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:23.255549 19623 raft_consensus.cc:504] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.256352 19623 leader_election.cc:288] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c4bd0f096b04c0690132264a36808b4 (127.16.235.66:33371), 25e5f01417ca45d5b106c1ecc4e78cd8 (127.16.235.67:36833)
I20260806 22:37:23.262779 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.256763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:60421 (local address 127.16.235.66:33371)
0806 22:37:23.256877 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.256880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.257006 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:37:23.257139 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.257140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.257167 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.257275 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:23.257595 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.258803 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.261244 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.262119 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.262265 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.262281 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.262549 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.262619 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:37:23.262779 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.257113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:40577 (local address 127.16.235.67:36833)
0806 22:37:23.257240 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.257242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.257302 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:37:23.257446 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.257447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.257456 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.257519 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:23.257721 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.258761 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.261245 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.262119 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.262265 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.262281 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.262598 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.262685 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:37:23.262841 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.256957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:36833 (local address 127.16.235.65:40577)
0806 22:37:23.257158 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.257182 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.257309 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.257527 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.257528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.257530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:23.257606 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.257607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.258900 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.258902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.261064 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.261069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.262274 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.262279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.262421 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.262428 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.262543 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.262633 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":91,"thread_start_us":114,"threads_started":1}
I20260806 22:37:23.262876 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.256605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:33371 (local address 127.16.235.65:60421)
0806 22:37:23.256901 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.256919 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.257036 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.257339 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.257350 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.257353 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:23.257474 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.257477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.258931 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.258931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.261064 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.261069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.262270 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.262271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.262421 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.262430 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.262543 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.262633 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":76,"thread_start_us":165,"threads_started":1}
I20260806 22:37:23.263787 19334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776cb33d3694f38b5d12588053d0fbc" candidate_uuid: "0fd9c07544bc474b91162e1cd6604f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4bd0f096b04c0690132264a36808b4" is_pre_election: true
I20260806 22:37:23.263892 19517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776cb33d3694f38b5d12588053d0fbc" candidate_uuid: "0fd9c07544bc474b91162e1cd6604f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" is_pre_election: true
I20260806 22:37:23.264115 19517 raft_consensus.cc:2454] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fd9c07544bc474b91162e1cd6604f2b in term 0.
I20260806 22:37:23.264115 19334 raft_consensus.cc:2454] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fd9c07544bc474b91162e1cd6604f2b in term 0.
I20260806 22:37:23.264495 19025 leader_election.cc:302] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd9c07544bc474b91162e1cd6604f2b, 1c4bd0f096b04c0690132264a36808b4; no voters:
I20260806 22:37:23.264761 19623 raft_consensus.cc:2781] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:23.264858 19623 raft_consensus.cc:482] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:23.264894 19623 raft_consensus.cc:3037] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:23.266088 19623 raft_consensus.cc:504] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.266340 19623 leader_election.cc:288] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [CANDIDATE]: Term 1 election: Requested vote from peers 1c4bd0f096b04c0690132264a36808b4 (127.16.235.66:33371), 25e5f01417ca45d5b106c1ecc4e78cd8 (127.16.235.67:36833)
I20260806 22:37:23.266574 19334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776cb33d3694f38b5d12588053d0fbc" candidate_uuid: "0fd9c07544bc474b91162e1cd6604f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4bd0f096b04c0690132264a36808b4"
I20260806 22:37:23.266821 19517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776cb33d3694f38b5d12588053d0fbc" candidate_uuid: "0fd9c07544bc474b91162e1cd6604f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8"
I20260806 22:37:23.266873 19334 raft_consensus.cc:3037] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:23.266988 19517 raft_consensus.cc:3037] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:23.268092 19334 raft_consensus.cc:2454] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fd9c07544bc474b91162e1cd6604f2b in term 1.
I20260806 22:37:23.268106 19517 raft_consensus.cc:2454] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fd9c07544bc474b91162e1cd6604f2b in term 1.
I20260806 22:37:23.268366 19025 leader_election.cc:302] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd9c07544bc474b91162e1cd6604f2b, 25e5f01417ca45d5b106c1ecc4e78cd8; no voters:
I20260806 22:37:23.268524 19623 raft_consensus.cc:2781] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:23.268784 19623 raft_consensus.cc:686] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 1 LEADER]: Becoming Leader. State: Replica: 0fd9c07544bc474b91162e1cd6604f2b, State: Running, Role: LEADER
I20260806 22:37:23.268954 19623 consensus_queue.cc:237] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.270396 18894 catalog_manager.cc:5495] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fd9c07544bc474b91162e1cd6604f2b (127.16.235.65). New cstate: current_term: 1 leader_uuid: "0fd9c07544bc474b91162e1cd6604f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:23.301641 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.297069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.16.235.65:35027)
0806 22:37:23.297153 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.297155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.297166 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:23.297370 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.297376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.297406 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.297529 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:23.297826 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.298563 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.299880 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.300151 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.300308 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.300793 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:23.301042 (+ 249us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:23.301279 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.301332 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.301382 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:37:23.306465 19334 raft_consensus.cc:1261] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [term 1 FOLLOWER]: Refusing update from remote peer 0fd9c07544bc474b91162e1cd6604f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:23.307278 19630 consensus_queue.cc:1035] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:23.308741 19517 raft_consensus.cc:1261] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Refusing update from remote peer 0fd9c07544bc474b91162e1cd6604f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:23.309427 19623 consensus_queue.cc:1035] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:23.318033 19634 mvcc.cc:205] Tried to move back new op lower bound from 7315684734176501760 to 7315684734030852096. Current Snapshot: MvccSnapshot[applied={T|T < 7315684734176501760}]
I20260806 22:37:23.319733 19633 mvcc.cc:205] Tried to move back new op lower bound from 7315684734176501760 to 7315684734030852096. Current Snapshot: MvccSnapshot[applied={T|T < 7315684734176501760}]
I20260806 22:37:23.332111 18894 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:23.332525 18894 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:23.335412 18894 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401: Sending DeleteTablet for 3 replicas of tablet b776cb33d3694f38b5d12588053d0fbc
I20260806 22:37:23.336879 19496 tablet_service.cc:1514] Processing DeleteTablet for tablet b776cb33d3694f38b5d12588053d0fbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:23 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:54928
I20260806 22:37:23.336861 19302 tablet_service.cc:1514] Processing DeleteTablet for tablet b776cb33d3694f38b5d12588053d0fbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:23 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:60640
I20260806 22:37:23.337016 19093 tablet_service.cc:1514] Processing DeleteTablet for tablet b776cb33d3694f38b5d12588053d0fbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:23 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:52934
I20260806 22:37:23.337819 19645 tablet_replica.cc:330] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: stopping tablet replica
I20260806 22:37:23.337970 19644 tablet_replica.cc:330] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: stopping tablet replica
I20260806 22:37:23.338073 19645 raft_consensus.cc:2229] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:23.338258 19644 raft_consensus.cc:2229] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:23.338598 19644 raft_consensus.cc:2258] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:23.338582 19645 raft_consensus.cc:2258] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:23.339323 19646 tablet_replica.cc:330] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: stopping tablet replica
I20260806 22:37:23.339638 19646 raft_consensus.cc:2229] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:23.339926 19644 ts_tablet_manager.cc:1905] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:23.340008 18894 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:23.340513 18894 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:23.340693 19646 raft_consensus.cc:2258] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:23.343721 19496 tablet_service.cc:1467] Processing CreateTablet for tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d (DEFAULT_TABLE table=test-table [id=4ef9f3c4a9a245579a4da16538a05045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:23.343721 19302 tablet_service.cc:1467] Processing CreateTablet for tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d (DEFAULT_TABLE table=test-table [id=4ef9f3c4a9a245579a4da16538a05045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:23.344108 19496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:23.344159 19302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:23.345485 19644 ts_tablet_manager.cc:1918] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:23.345816 19644 log.cc:1196] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/b776cb33d3694f38b5d12588053d0fbc
I20260806 22:37:23.346863 19644 ts_tablet_manager.cc:1939] T b776cb33d3694f38b5d12588053d0fbc P 25e5f01417ca45d5b106c1ecc4e78cd8: Deleting consensus metadata
I20260806 22:37:23.347854 19093 tablet_service.cc:1467] Processing CreateTablet for tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d (DEFAULT_TABLE table=test-table [id=4ef9f3c4a9a245579a4da16538a05045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:23.348229 19093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:23.348731 19615 tablet_bootstrap.cc:492] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: Bootstrap starting.
I20260806 22:37:23.350217 19615 tablet_bootstrap.cc:654] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:23.351104 18872 catalog_manager.cc:4841] TS 25e5f01417ca45d5b106c1ecc4e78cd8 (127.16.235.67:36833): tablet b776cb33d3694f38b5d12588053d0fbc (table test-table [id=fc58f618318a4145aae910a7d9669d2f]) successfully deleted
I20260806 22:37:23.356235 19615 tablet_bootstrap.cc:492] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: No bootstrap required, opened a new log
I20260806 22:37:23.356477 19615 ts_tablet_manager.cc:1397] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 22:37:23.356983 19615 raft_consensus.cc:348] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.357156 19615 raft_consensus.cc:374] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:23.357203 19615 raft_consensus.cc:729] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e5f01417ca45d5b106c1ecc4e78cd8, State: Initialized, Role: FOLLOWER
I20260806 22:37:23.357354 19615 consensus_queue.cc:260] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.357780 19614 tablet_bootstrap.cc:492] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: Bootstrap starting.
I20260806 22:37:23.358120 19615 ts_tablet_manager.cc:1428] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 22:37:23.358765 19616 tablet_bootstrap.cc:492] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: Bootstrap starting.
I20260806 22:37:23.358968 19614 tablet_bootstrap.cc:654] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:23.359679 19616 tablet_bootstrap.cc:654] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:23.360472 19614 tablet_bootstrap.cc:492] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: No bootstrap required, opened a new log
I20260806 22:37:23.360574 19614 ts_tablet_manager.cc:1397] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:37:23.360867 19614 raft_consensus.cc:348] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.361027 19614 raft_consensus.cc:374] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:23.361078 19614 raft_consensus.cc:729] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd9c07544bc474b91162e1cd6604f2b, State: Initialized, Role: FOLLOWER
I20260806 22:37:23.361153 19614 consensus_queue.cc:260] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.361562 19614 ts_tablet_manager.cc:1428] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:23.360875 19645 ts_tablet_manager.cc:1905] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:23.363351 19646 ts_tablet_manager.cc:1905] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:23.365760 19645 ts_tablet_manager.cc:1918] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:23.365926 19645 log.cc:1196] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/b776cb33d3694f38b5d12588053d0fbc
I20260806 22:37:23.366832 19645 ts_tablet_manager.cc:1939] T b776cb33d3694f38b5d12588053d0fbc P 0fd9c07544bc474b91162e1cd6604f2b: Deleting consensus metadata
I20260806 22:37:23.368220 18869 catalog_manager.cc:4841] TS 0fd9c07544bc474b91162e1cd6604f2b (127.16.235.65:35027): tablet b776cb33d3694f38b5d12588053d0fbc (table test-table [id=fc58f618318a4145aae910a7d9669d2f]) successfully deleted
I20260806 22:37:23.369262 19646 ts_tablet_manager.cc:1918] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:23.369567 19646 log.cc:1196] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/b776cb33d3694f38b5d12588053d0fbc
I20260806 22:37:23.370129 19646 ts_tablet_manager.cc:1939] T b776cb33d3694f38b5d12588053d0fbc P 1c4bd0f096b04c0690132264a36808b4: Deleting consensus metadata
I20260806 22:37:23.371387 18872 catalog_manager.cc:4841] TS 1c4bd0f096b04c0690132264a36808b4 (127.16.235.66:33371): tablet b776cb33d3694f38b5d12588053d0fbc (table test-table [id=fc58f618318a4145aae910a7d9669d2f]) successfully deleted
I20260806 22:37:23.372009 19616 tablet_bootstrap.cc:492] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: No bootstrap required, opened a new log
I20260806 22:37:23.372139 19616 ts_tablet_manager.cc:1397] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260806 22:37:23.372635 19616 raft_consensus.cc:348] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.372913 19616 raft_consensus.cc:374] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:23.373018 19616 raft_consensus.cc:729] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4bd0f096b04c0690132264a36808b4, State: Initialized, Role: FOLLOWER
I20260806 22:37:23.373247 19616 consensus_queue.cc:260] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.373523 19616 ts_tablet_manager.cc:1428] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260806 22:37:23.376770 19579 tablet.cc:2369] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:23.386864 19625 raft_consensus.cc:482] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:23.387061 19625 raft_consensus.cc:504] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.387681 19625 leader_election.cc:288] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fd9c07544bc474b91162e1cd6604f2b (127.16.235.65:35027), 25e5f01417ca45d5b106c1ecc4e78cd8 (127.16.235.67:36833)
I20260806 22:37:23.394194 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.387970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:35027 (local address 127.16.235.66:48473)
0806 22:37:23.388075 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.388101 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.388199 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.388675 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.388676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.388688 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:23.388804 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.388811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.390256 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.390267 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.392347 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.392351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.393724 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.393732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.393862 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.393876 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.393953 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.394032 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":109}
I20260806 22:37:23.394863 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.388392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:49745 (local address 127.16.235.67:36833)
0806 22:37:23.388470 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.388472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.388662 (+ 190us) server_negotiation.cc:407] Connection header received
0806 22:37:23.388861 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.388862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.388900 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.388953 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:23.389430 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.390443 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.392836 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.393850 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.393940 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.393951 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.394611 (+ 660us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.394714 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:37:23.395439 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.387990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:36833 (local address 127.16.235.66:49745)
0806 22:37:23.388566 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.388578 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.388700 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.389083 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.389084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.389086 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:23.389245 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.389248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.390634 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.390635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.392562 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.392566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.394245 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.394246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.394413 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.394434 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.395108 (+ 674us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.395198 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":361,"thread_start_us":89,"threads_started":1}
I20260806 22:37:23.395925 19517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dfc35d0755f4d12b7d80e3e78e2fd3d" candidate_uuid: "1c4bd0f096b04c0690132264a36808b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" is_pre_election: true
I20260806 22:37:23.396166 19517 raft_consensus.cc:2454] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c4bd0f096b04c0690132264a36808b4 in term 0.
I20260806 22:37:23.396234 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.388271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:48473 (local address 127.16.235.65:35027)
0806 22:37:23.388378 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.388381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.388392 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:23.388443 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.388444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.388467 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.388582 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:23.388945 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.390067 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.392565 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.393515 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.393642 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.393655 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.396024 (+ 2369us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.396114 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 22:37:23.396597 19238 leader_election.cc:302] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c4bd0f096b04c0690132264a36808b4, 25e5f01417ca45d5b106c1ecc4e78cd8; no voters:
I20260806 22:37:23.396697 19123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dfc35d0755f4d12b7d80e3e78e2fd3d" candidate_uuid: "1c4bd0f096b04c0690132264a36808b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd9c07544bc474b91162e1cd6604f2b" is_pre_election: true
I20260806 22:37:23.396849 19625 raft_consensus.cc:2781] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:23.396919 19625 raft_consensus.cc:482] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:23.396950 19123 raft_consensus.cc:2454] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c4bd0f096b04c0690132264a36808b4 in term 0.
I20260806 22:37:23.396977 19625 raft_consensus.cc:3037] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:23.399045 19625 raft_consensus.cc:504] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.399379 19625 leader_election.cc:288] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [CANDIDATE]: Term 1 election: Requested vote from peers 0fd9c07544bc474b91162e1cd6604f2b (127.16.235.65:35027), 25e5f01417ca45d5b106c1ecc4e78cd8 (127.16.235.67:36833)
I20260806 22:37:23.399873 19123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dfc35d0755f4d12b7d80e3e78e2fd3d" candidate_uuid: "1c4bd0f096b04c0690132264a36808b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd9c07544bc474b91162e1cd6604f2b"
I20260806 22:37:23.399896 19517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dfc35d0755f4d12b7d80e3e78e2fd3d" candidate_uuid: "1c4bd0f096b04c0690132264a36808b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8"
I20260806 22:37:23.400023 19123 raft_consensus.cc:3037] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:23.400038 19517 raft_consensus.cc:3037] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:23.401825 19517 raft_consensus.cc:2454] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c4bd0f096b04c0690132264a36808b4 in term 1.
I20260806 22:37:23.401878 19123 raft_consensus.cc:2454] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c4bd0f096b04c0690132264a36808b4 in term 1.
W20260806 22:37:23.402376 19390 tablet.cc:2369] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:23.402783 19233 leader_election.cc:302] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd9c07544bc474b91162e1cd6604f2b, 1c4bd0f096b04c0690132264a36808b4; no voters:
I20260806 22:37:23.403031 19625 raft_consensus.cc:2781] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:23.403282 19625 raft_consensus.cc:686] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 1 LEADER]: Becoming Leader. State: Replica: 1c4bd0f096b04c0690132264a36808b4, State: Running, Role: LEADER
I20260806 22:37:23.403432 19625 consensus_queue.cc:237] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.405150 18894 catalog_manager.cc:5495] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c4bd0f096b04c0690132264a36808b4 (127.16.235.66). New cstate: current_term: 1 leader_uuid: "1c4bd0f096b04c0690132264a36808b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:37:23 dist-test-slave-3kk6 krb5kdc[18834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:23.460417 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.444252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40289 (local address 127.0.0.1:35204)
0806 22:37:23.444514 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.444534 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.444647 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.444990 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.444991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.445691 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:23.445776 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.445779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.447312 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.447313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.448606 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.448610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.449425 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.449426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.449515 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.450560 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:23.450567 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:23.453524 (+ 2957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:23.457520 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:23.457535 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:23.457549 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:23.458122 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:23.458592 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:23.458593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:23.458594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:23.458771 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:23.459635 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:23.459658 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:23.459777 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.459919 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.460179 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":152,"threads_started":1}
I20260806 22:37:23.461193 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.444300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35204 (local address 127.16.235.126:40289)
0806 22:37:23.444377 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.444380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.444640 (+ 260us) server_negotiation.cc:407] Connection header received
0806 22:37:23.444770 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.444771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.444785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.444911 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.445930 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.447139 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.448781 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.449266 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.449392 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.453718 (+ 4326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.453721 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.453721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.453736 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.457300 (+ 3564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.458315 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.458317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.458318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.458458 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.459174 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.459176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.459176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.459483 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.459655 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.460016 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.460384 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":133,"server-negotiator.queue_time_us":39}
I20260806 22:37:23.465801 18894 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35204:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:37:23.466295 18894 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:23.469121 19093 tablet_service.cc:1467] Processing CreateTablet for tablet 80e61df50ed744349435ef63598b6112 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff5a1fbdbb284a44ba12e8aaef49ee94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:37:23.469146 19496 tablet_service.cc:1467] Processing CreateTablet for tablet 80e61df50ed744349435ef63598b6112 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff5a1fbdbb284a44ba12e8aaef49ee94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:37:23.469534 19093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e61df50ed744349435ef63598b6112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:23.469534 19496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e61df50ed744349435ef63598b6112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:23.471005 19302 tablet_service.cc:1467] Processing CreateTablet for tablet 80e61df50ed744349435ef63598b6112 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff5a1fbdbb284a44ba12e8aaef49ee94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:37:23.471335 19302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e61df50ed744349435ef63598b6112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:23.472391 19615 tablet_bootstrap.cc:492] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: Bootstrap starting.
I20260806 22:37:23.473042 19615 tablet_bootstrap.cc:654] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:23.474066 19614 tablet_bootstrap.cc:492] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: Bootstrap starting.
I20260806 22:37:23.474320 19616 tablet_bootstrap.cc:492] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: Bootstrap starting.
I20260806 22:37:23.475040 19616 tablet_bootstrap.cc:654] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:23.475371 19614 tablet_bootstrap.cc:654] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:23.475383 19615 tablet_bootstrap.cc:492] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: No bootstrap required, opened a new log
I20260806 22:37:23.475479 19615 ts_tablet_manager.cc:1397] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 22:37:23.475898 19615 raft_consensus.cc:348] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.476104 19615 raft_consensus.cc:374] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:23.476189 19615 raft_consensus.cc:729] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e5f01417ca45d5b106c1ecc4e78cd8, State: Initialized, Role: FOLLOWER
I20260806 22:37:23.476280 19615 consensus_queue.cc:260] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.476500 19615 ts_tablet_manager.cc:1428] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:37:23.476666 19622 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.476858 19622 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:23.477612 19616 tablet_bootstrap.cc:492] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: No bootstrap required, opened a new log
I20260806 22:37:23.477720 19616 ts_tablet_manager.cc:1397] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:37:23.477962 19616 raft_consensus.cc:348] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.478116 19616 raft_consensus.cc:374] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:23.478161 19616 raft_consensus.cc:729] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4bd0f096b04c0690132264a36808b4, State: Initialized, Role: FOLLOWER
I20260806 22:37:23.478261 19616 consensus_queue.cc:260] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.478425 19616 ts_tablet_manager.cc:1428] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:37:23.478503 19625 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.479066 19625 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:23.480919 19614 tablet_bootstrap.cc:492] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: No bootstrap required, opened a new log
I20260806 22:37:23.481025 19614 ts_tablet_manager.cc:1397] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260806 22:37:23.481261 19614 raft_consensus.cc:348] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.481352 19614 raft_consensus.cc:374] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:23.481383 19614 raft_consensus.cc:729] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd9c07544bc474b91162e1cd6604f2b, State: Initialized, Role: FOLLOWER
I20260806 22:37:23.481474 19614 consensus_queue.cc:260] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.481695 19623 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.481889 19623 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:23.481712 19614 ts_tablet_manager.cc:1428] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:37:23.568145 19623 raft_consensus.cc:482] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:23.568334 19623 raft_consensus.cc:504] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.568756 19623 leader_election.cc:288] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c4bd0f096b04c0690132264a36808b4 (127.16.235.66:33371), 25e5f01417ca45d5b106c1ecc4e78cd8 (127.16.235.67:36833)
I20260806 22:37:23.569257 19517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e61df50ed744349435ef63598b6112" candidate_uuid: "0fd9c07544bc474b91162e1cd6604f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" is_pre_election: true
I20260806 22:37:23.569263 19334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e61df50ed744349435ef63598b6112" candidate_uuid: "0fd9c07544bc474b91162e1cd6604f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4bd0f096b04c0690132264a36808b4" is_pre_election: true
I20260806 22:37:23.569445 19517 raft_consensus.cc:2454] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fd9c07544bc474b91162e1cd6604f2b in term 0.
I20260806 22:37:23.569481 19334 raft_consensus.cc:2454] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fd9c07544bc474b91162e1cd6604f2b in term 0.
I20260806 22:37:23.569796 19025 leader_election.cc:302] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd9c07544bc474b91162e1cd6604f2b, 1c4bd0f096b04c0690132264a36808b4; no voters:
I20260806 22:37:23.569999 19623 raft_consensus.cc:2781] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:23.570087 19623 raft_consensus.cc:482] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:23.570138 19623 raft_consensus.cc:3037] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:23.571554 19623 raft_consensus.cc:504] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.571835 19623 leader_election.cc:288] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [CANDIDATE]: Term 1 election: Requested vote from peers 1c4bd0f096b04c0690132264a36808b4 (127.16.235.66:33371), 25e5f01417ca45d5b106c1ecc4e78cd8 (127.16.235.67:36833)
I20260806 22:37:23.572295 19517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e61df50ed744349435ef63598b6112" candidate_uuid: "0fd9c07544bc474b91162e1cd6604f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8"
I20260806 22:37:23.572343 19334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e61df50ed744349435ef63598b6112" candidate_uuid: "0fd9c07544bc474b91162e1cd6604f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4bd0f096b04c0690132264a36808b4"
I20260806 22:37:23.572453 19517 raft_consensus.cc:3037] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:23.572476 19334 raft_consensus.cc:3037] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:23.574246 19334 raft_consensus.cc:2454] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fd9c07544bc474b91162e1cd6604f2b in term 1.
I20260806 22:37:23.574249 19517 raft_consensus.cc:2454] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fd9c07544bc474b91162e1cd6604f2b in term 1.
I20260806 22:37:23.574651 19025 leader_election.cc:302] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd9c07544bc474b91162e1cd6604f2b, 1c4bd0f096b04c0690132264a36808b4; no voters:
I20260806 22:37:23.574954 19623 raft_consensus.cc:2781] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:23.575095 19623 raft_consensus.cc:686] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [term 1 LEADER]: Becoming Leader. State: Replica: 0fd9c07544bc474b91162e1cd6604f2b, State: Running, Role: LEADER
I20260806 22:37:23.575222 19623 consensus_queue.cc:237] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } }
I20260806 22:37:23.575918 19630 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: TxnStatusTablet state changed. Reason: New leader 0fd9c07544bc474b91162e1cd6604f2b. Latest consensus state: current_term: 1 leader_uuid: "0fd9c07544bc474b91162e1cd6604f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.576140 19630 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:37:23.576735 19669 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:37:23.576732 18888 catalog_manager.cc:5495] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fd9c07544bc474b91162e1cd6604f2b (127.16.235.65). New cstate: current_term: 1 leader_uuid: "0fd9c07544bc474b91162e1cd6604f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:23.576947 19669 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:37:23.606319 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.601462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:35027 (local address 127.0.0.1:52956)
0806 22:37:23.601736 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.601767 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.601864 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.602105 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.602105 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.602107 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:23.602192 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.602194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.603635 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.603637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.604762 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.604766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.605559 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.605560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.605647 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.605676 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:23.606055 (+ 379us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:23.606062 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.606128 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.606222 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260806 22:37:23.606320 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.601611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.16.235.65:35027)
0806 22:37:23.601707 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.601710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.601858 (+ 148us) server_negotiation.cc:407] Connection header received
0806 22:37:23.601972 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.601973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.601987 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.602103 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:23.602298 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.603478 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.604916 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.605390 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.605563 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.605872 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:23.605951 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:23.606047 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.606150 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.606193 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:37:23.608999 19517 raft_consensus.cc:1261] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Refusing update from remote peer 0fd9c07544bc474b91162e1cd6604f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:23.609002 19334 raft_consensus.cc:1261] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4 [term 1 FOLLOWER]: Refusing update from remote peer 0fd9c07544bc474b91162e1cd6604f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:23.609405 19630 consensus_queue.cc:1035] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:23.609594 19623 consensus_queue.cc:1035] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:23.610859 19622 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: TxnStatusTablet state changed. Reason: New leader 0fd9c07544bc474b91162e1cd6604f2b. Latest consensus state: current_term: 1 leader_uuid: "0fd9c07544bc474b91162e1cd6604f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.611299 19622 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:23.611166 19625 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: TxnStatusTablet state changed. Reason: New leader 0fd9c07544bc474b91162e1cd6604f2b. Latest consensus state: current_term: 1 leader_uuid: "0fd9c07544bc474b91162e1cd6604f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.611712 19625 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:23.614578 19630 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fd9c07544bc474b91162e1cd6604f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.614933 19630 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:37:23.617663 19623 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0fd9c07544bc474b91162e1cd6604f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.618034 19623 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 0fd9c07544bc474b91162e1cd6604f2b: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:37:23.619719 19635 mvcc.cc:205] Tried to move back new op lower bound from 7315684735419170816 to 7315684735284940800. Current Snapshot: MvccSnapshot[applied={T|T < 7315684735419170816}]
I20260806 22:37:23.620353 19625 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fd9c07544bc474b91162e1cd6604f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.620893 19622 tablet_replica.cc:439] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0fd9c07544bc474b91162e1cd6604f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 } } peers { permanent_uuid: "1c4bd0f096b04c0690132264a36808b4" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33371 } } peers { permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 } } }
I20260806 22:37:23.621112 19625 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 1c4bd0f096b04c0690132264a36808b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:23.621225 19622 tablet_replica.cc:442] T 80e61df50ed744349435ef63598b6112 P 25e5f01417ca45d5b106c1ecc4e78cd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:23.640020 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.626158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.16.235.66:33371)
0806 22:37:23.626371 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.626374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.626415 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:37:23.626736 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.626746 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.626825 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.627267 (+ 442us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:23.627925 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.630026 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.634280 (+ 4254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.635262 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.635739 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.636494 (+ 755us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:23.636966 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:23.639277 (+ 2311us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.639411 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.639542 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260806 22:37:23.667230 19676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.653649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:35027 (local address 127.16.235.66:37275)
0806 22:37:23.654277 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.654355 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.654794 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.655802 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.655805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.655808 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:23.656401 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.656425 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.660315 (+ 3890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.660325 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.662070 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.662078 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.665226 (+ 3148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.665229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.665399 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.665474 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:23.666470 (+ 996us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:23.666479 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.666620 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.666965 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":341,"threads_started":1}
I20260806 22:37:23.667146 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.654286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:37275 (local address 127.16.235.65:35027)
0806 22:37:23.654515 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.654527 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.654603 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:37:23.655468 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.655470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.655545 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.655736 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:23.656905 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.659213 (+ 2308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.662356 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.664002 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.664358 (+ 356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.665801 (+ 1443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:23.666035 (+ 234us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:23.666479 (+ 444us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.666684 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.666743 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260806 22:37:23.677935 19517 raft_consensus.cc:1261] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Refusing update from remote peer 1c4bd0f096b04c0690132264a36808b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:23.678117 19123 raft_consensus.cc:1261] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [term 1 FOLLOWER]: Refusing update from remote peer 1c4bd0f096b04c0690132264a36808b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:23.678794 19652 consensus_queue.cc:1035] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e5f01417ca45d5b106c1ecc4e78cd8" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 36833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:23.679243 19625 consensus_queue.cc:1035] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fd9c07544bc474b91162e1cd6604f2b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:23.716768 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.707278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:53979 (local address 127.16.235.66:33371)
0806 22:37:23.707405 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.707413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.708350 (+ 937us) server_negotiation.cc:407] Connection header received
0806 22:37:23.708771 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.708775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.708801 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.708988 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:23.709352 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.710430 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.714088 (+ 3658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.714984 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.715279 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.716072 (+ 793us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:23.716217 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:23.716346 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.716638 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.716680 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:37:23.716917 19682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.707496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:33371 (local address 127.16.235.65:53979)
0806 22:37:23.708081 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:23.708145 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:23.708393 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:23.709005 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:23.709011 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:23.709013 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:23.709162 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.709164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.710820 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.710826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.713639 (+ 2813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:23.713682 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.715286 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.715288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:23.715516 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.715615 (+ 99us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:23.716398 (+ 783us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:23.716433 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:37:23.716635 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 22:37:23.716745 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":318,"threads_started":1}
I20260806 22:37:23.743884 18894 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:23.744230 18894 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:23.746066 18894 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 800924947cdd471d87c0ecb4a3b6f401: Sending DeleteTablet for 3 replicas of tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d
I20260806 22:37:23.746791 19302 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:23 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:60640
I20260806 22:37:23.746884 19496 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:23 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:54928
I20260806 22:37:23.747009 19646 tablet_replica.cc:330] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: stopping tablet replica
I20260806 22:37:23.747040 19644 tablet_replica.cc:330] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: stopping tablet replica
I20260806 22:37:23.747082 19646 raft_consensus.cc:2229] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:23.747133 19644 raft_consensus.cc:2229] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:23.747262 19644 raft_consensus.cc:2258] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:23.747390 19646 raft_consensus.cc:2258] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:23.747510 19093 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:23 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:52934
I20260806 22:37:23.748160 19645 tablet_replica.cc:330] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: stopping tablet replica
I20260806 22:37:23.748315 19645 raft_consensus.cc:2229] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:23.748492 19645 raft_consensus.cc:2258] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:23.748941 19646 ts_tablet_manager.cc:1905] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:23.749770 19645 ts_tablet_manager.cc:1905] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:23.751214 19646 ts_tablet_manager.cc:1918] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:23.751272 19646 log.cc:1196] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/9dfc35d0755f4d12b7d80e3e78e2fd3d
I20260806 22:37:23.751417 19645 ts_tablet_manager.cc:1918] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:23.751470 19645 log.cc:1196] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/9dfc35d0755f4d12b7d80e3e78e2fd3d
I20260806 22:37:23.751513 19646 ts_tablet_manager.cc:1939] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 1c4bd0f096b04c0690132264a36808b4: Deleting consensus metadata
I20260806 22:37:23.751703 19645 ts_tablet_manager.cc:1939] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 0fd9c07544bc474b91162e1cd6604f2b: Deleting consensus metadata
I20260806 22:37:23.752049 18869 catalog_manager.cc:4841] TS 0fd9c07544bc474b91162e1cd6604f2b (127.16.235.65:35027): tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d (table test-table [id=4ef9f3c4a9a245579a4da16538a05045]) successfully deleted
I20260806 22:37:23.752084 19644 ts_tablet_manager.cc:1905] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:23.752393 18872 catalog_manager.cc:4841] TS 1c4bd0f096b04c0690132264a36808b4 (127.16.235.66:33371): tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d (table test-table [id=4ef9f3c4a9a245579a4da16538a05045]) successfully deleted
I20260806 22:37:23.753829 19644 ts_tablet_manager.cc:1918] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:23.753917 19644 log.cc:1196] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/9dfc35d0755f4d12b7d80e3e78e2fd3d
I20260806 22:37:23.754156 19644 ts_tablet_manager.cc:1939] T 9dfc35d0755f4d12b7d80e3e78e2fd3d P 25e5f01417ca45d5b106c1ecc4e78cd8: Deleting consensus metadata
I20260806 22:37:23.754491 18872 catalog_manager.cc:4841] TS 25e5f01417ca45d5b106c1ecc4e78cd8 (127.16.235.67:36833): tablet 9dfc35d0755f4d12b7d80e3e78e2fd3d (table test-table [id=4ef9f3c4a9a245579a4da16538a05045]) successfully deleted
Aug 06 22:37:23 dist-test-slave-3kk6 krb5kdc[18834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:23.762459 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.752846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.16.235.65:35027)
0806 22:37:23.752889 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.752892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.753199 (+ 307us) server_negotiation.cc:407] Connection header received
0806 22:37:23.753308 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.753309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.753321 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.753352 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.754409 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.755039 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.755851 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.756059 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.756142 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.759370 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.759384 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.759384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.759393 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.761167 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.761575 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.761579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.761579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.761646 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.761856 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.761857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.761858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.762023 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.762187 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.762253 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.762346 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:37:23.762812 19184 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52964
I20260806 22:37:23.770267 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:23.764018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35218 (local address 127.16.235.126:40289)
0806 22:37:23.764072 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:23.764074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:23.764186 (+ 112us) server_negotiation.cc:407] Connection header received
0806 22:37:23.764253 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:23.764254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:23.764266 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:23.764322 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:23.764837 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.765352 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.766044 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:23.766260 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:23.766350 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:23.767612 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:23.767614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:23.767614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:23.767623 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:23.769103 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.769470 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.769471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.769471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.769531 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:23.769740 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:23.769740 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:23.769741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:23.769855 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:23.769974 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:23.770068 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:37:23.770152 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 22:37:23.770614 18894 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35218
I20260806 22:37:23.771450 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 18997
I20260806 22:37:23.808710 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 19205
I20260806 22:37:23.824025 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 19407
I20260806 22:37:23.838881 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 18849
2026-08-06T22:37:23Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2823 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:23 dist-test-slave-3kk6 krb5kdc[19695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:23 dist-test-slave-3kk6 krb5kdc[19695](info): set up 2 sockets
Aug 06 22:37:23 dist-test-slave-3kk6 krb5kdc[19695](info): commencing operation
krb5kdc: starting...
W20260806 22:37:25.072996 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:25.205828 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45091
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:45091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:25.289733 19712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:25.289996 19712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:25.290035 19712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:25.291277 19712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:25.291347 19712 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:37:25.291366 19712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:25.291388 19712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:25.291420 19712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:25.291445 19712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:25.292970 19712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:45091
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:25.293262 19712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:25.293503 19712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:25.298593 19718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:25.298599 19724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:25.298936 19712 server_base.cc:1034] running on GCE node
W20260806 22:37:25.298600 19717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:25.299353 19712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:25.299660 19712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:25.300815 19712 hybrid_clock.cc:648] HybridClock initialized: now 1786055845300805 us; error 36 us; skew 500 ppm
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:25.303653 19712 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:25.304193 19712 webserver.cc:460] Webserver started at http://127.16.235.126:41563/ using document root <none> and password file <none>
I20260806 22:37:25.304430 19712 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:25.304476 19712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:25.304652 19712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:25.305794 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "a5c990f0635d420a96f3a83f4b93255a"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "6464f6012b3942c8103e19e9bf9e4d7a"
server_key_iv: "4b6c7cdad5fc6c6455dd5c96985350ef"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.306181 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "a5c990f0635d420a96f3a83f4b93255a"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "6464f6012b3942c8103e19e9bf9e4d7a"
server_key_iv: "4b6c7cdad5fc6c6455dd5c96985350ef"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.307660 19712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:25.308452 19733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.308635 19712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:37:25.308736 19712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "a5c990f0635d420a96f3a83f4b93255a"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "6464f6012b3942c8103e19e9bf9e4d7a"
server_key_iv: "4b6c7cdad5fc6c6455dd5c96985350ef"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.308813 19712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:25.316239 19712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:25.318620 19712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:25.318856 19712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:25.324132 19712 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:45091
I20260806 22:37:25.324222 19825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:45091 every 8 connection(s)
I20260806 22:37:25.324561 19712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:25.325268 19826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:25.327775 19826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a: Bootstrap starting.
I20260806 22:37:25.328398 19826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:25.328738 19826 log.cc:824] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:25.329492 19826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a: No bootstrap required, opened a new log
I20260806 22:37:25.331899 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 19712
I20260806 22:37:25.331986 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:25.331948 19826 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c990f0635d420a96f3a83f4b93255a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45091 } }
I20260806 22:37:25.332177 17325 external_mini_cluster.cc:1428] Setting key 4e4edc2b011368e23a1433c395b46750
I20260806 22:37:25.332185 19826 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:25.332239 19826 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5c990f0635d420a96f3a83f4b93255a, State: Initialized, Role: FOLLOWER
I20260806 22:37:25.332427 19826 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c990f0635d420a96f3a83f4b93255a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45091 } }
I20260806 22:37:25.332540 19826 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:25.332587 19826 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:25.332645 19826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:25.333426 19826 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c990f0635d420a96f3a83f4b93255a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45091 } }
I20260806 22:37:25.333552 19826 leader_election.cc:302] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5c990f0635d420a96f3a83f4b93255a; no voters:
I20260806 22:37:25.333763 19826 leader_election.cc:288] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:25.333783 19832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:25.333947 19832 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [term 1 LEADER]: Becoming Leader. State: Replica: a5c990f0635d420a96f3a83f4b93255a, State: Running, Role: LEADER
I20260806 22:37:25.334061 19832 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c990f0635d420a96f3a83f4b93255a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45091 } }
I20260806 22:37:25.334091 19826 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:25.334399 19832 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5c990f0635d420a96f3a83f4b93255a. Latest consensus state: current_term: 1 leader_uuid: "a5c990f0635d420a96f3a83f4b93255a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c990f0635d420a96f3a83f4b93255a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45091 } } }
I20260806 22:37:25.334488 19832 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:25.334417 19834 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5c990f0635d420a96f3a83f4b93255a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5c990f0635d420a96f3a83f4b93255a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45091 } } }
I20260806 22:37:25.334830 19834 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:25.335503 19842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:25.336023 19842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:25.338008 19842 catalog_manager.cc:1347] Generated new cluster ID: c3d481aeda1d4c4e83016f60bf8b49bc
I20260806 22:37:25.338066 19842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:25.344698 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.332798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.16.235.126:45091)
0806 22:37:25.332997 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.333007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.333043 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:37:25.333142 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.333148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.333191 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.333312 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:25.334221 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.335823 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.336664 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.337311 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.337427 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.340597 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:25.340601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:25.340601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:25.340612 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:25.343029 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.343426 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.343429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.343430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.343551 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.343772 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.343773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.343773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.344030 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:25.344238 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.344315 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.344468 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260806 22:37:25.370847 19842 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:25.371469 19842 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:25.378686 19842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a: Generated new TSK 0
I20260806 22:37:25.378943 19842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:25.413699 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45091
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:25.522918 19860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:25.523141 19860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:25.523167 19860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:25.524329 19860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:25.524376 19860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:25.524396 19860 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:37:25.524408 19860 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:37:25.524426 19860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:25.525966 19860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.235.126:45091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:25.526223 19860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:25.526479 19860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:37:25.531685 19860 server_base.cc:1034] running on GCE node
W20260806 22:37:25.531625 19869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:25.531697 19871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:25.531642 19865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:25.532076 19860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:25.532349 19860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:25.533496 19860 hybrid_clock.cc:648] HybridClock initialized: now 1786055845533474 us; error 45 us; skew 500 ppm
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:25.536369 19860 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:25.536831 19860 webserver.cc:460] Webserver started at http://127.16.235.65:37823/ using document root <none> and password file <none>
I20260806 22:37:25.537071 19860 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:25.537118 19860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:25.537283 19860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:25.538396 19860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "e58170824e82419d9c5b35917150641b"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "b6d8fa38badc37c02973981689c1ba12"
server_key_iv: "cb0abb766ea93135321ee6d23551d2aa"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.538828 19860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "e58170824e82419d9c5b35917150641b"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "b6d8fa38badc37c02973981689c1ba12"
server_key_iv: "cb0abb766ea93135321ee6d23551d2aa"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.540288 19860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:25.541069 19883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.541249 19860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.541360 19860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "e58170824e82419d9c5b35917150641b"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "b6d8fa38badc37c02973981689c1ba12"
server_key_iv: "cb0abb766ea93135321ee6d23551d2aa"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.541437 19860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:25.550688 19860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:25.553972 19860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:25.554152 19860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:25.554910 19860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:25.554975 19860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.555019 19860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:25.555040 19860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.562821 19860 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:36347
I20260806 22:37:25.562885 20044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:36347 every 8 connection(s)
I20260806 22:37:25.563231 19860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:25.566470 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.555750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:56689 (local address 127.16.235.126:45091)
0806 22:37:25.555831 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.555834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.556121 (+ 287us) server_negotiation.cc:407] Connection header received
0806 22:37:25.556717 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.556718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.556737 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.556797 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:25.557733 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.558614 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.560422 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.560801 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.560902 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.563978 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:25.563980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:25.563980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:25.563987 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:25.565260 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.565647 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.565648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.565649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.565700 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.565922 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.565923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.565923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.566032 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:25.566083 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.566327 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.566408 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:37:25.566629 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.555379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45091 (local address 127.16.235.65:56689)
0806 22:37:25.555967 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.556018 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.556593 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.556875 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.556879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.557279 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:25.557582 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.557588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.558807 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.558808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.560310 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.560314 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.560926 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.560927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.561100 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.561880 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:25.561891 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:25.563865 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:25.565337 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.565344 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.565346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.565577 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.565761 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.565762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.565762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.565867 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.566087 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:25.566091 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:25.566212 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.566298 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.566427 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":106,"threads_started":1}
I20260806 22:37:25.567338 20045 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45091
I20260806 22:37:25.567428 20045 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:25.567625 20045 heartbeater.cc:511] Master 127.16.235.126:45091 requested a full tablet report, sending...
I20260806 22:37:25.568243 19755 ts_manager.cc:194] Registered new tserver with Master: e58170824e82419d9c5b35917150641b (127.16.235.65:36347)
I20260806 22:37:25.569106 19755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:56689
I20260806 22:37:25.571005 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 19860
I20260806 22:37:25.571115 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:25.571346 17325 external_mini_cluster.cc:1428] Setting key 9cf2d01290f61dea0359b23ca3eb9038
I20260806 22:37:25.574586 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.568513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45091 (local address 127.16.235.65:50949)
0806 22:37:25.568653 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.568664 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.568727 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.568841 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.568842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.569003 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:25.569060 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.569062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.569920 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.569920 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.570421 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.570422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.570878 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.570879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.570939 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.571473 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:25.571478 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:25.571862 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:25.573527 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.573528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.573529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.573771 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.574005 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.574006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.574006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.574064 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.574348 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:25.574349 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:25.574386 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.574438 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.574519 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260806 22:37:25.574702 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.568582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:50949 (local address 127.16.235.126:45091)
0806 22:37:25.568625 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.568627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.568734 (+ 107us) server_negotiation.cc:407] Connection header received
0806 22:37:25.568762 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.568763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.568773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.568839 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:25.569129 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.569820 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.570502 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.570782 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.570879 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.571934 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:25.571935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:25.571935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:25.571942 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:25.573442 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.573860 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.573861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.573862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.573929 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.574129 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.574130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.574131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.574282 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:25.574354 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.574466 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.574552 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:25.601356 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45091
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:25.689345 20055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:25.689586 20055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:25.689616 20055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:25.690846 20055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:25.690891 20055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:25.690912 20055 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:37:25.690924 20055 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:37:25.690941 20055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:25.692422 20055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.235.126:45091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:25.692629 20055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:25.692838 20055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:25.698266 20066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:25.698246 20064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:25.698251 20061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:25.698479 20055 server_base.cc:1034] running on GCE node
I20260806 22:37:25.698748 20055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:25.699035 20055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:25.700214 20055 hybrid_clock.cc:648] HybridClock initialized: now 1786055845700187 us; error 43 us; skew 500 ppm
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:25.702901 20055 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:25.703404 20055 webserver.cc:460] Webserver started at http://127.16.235.66:37749/ using document root <none> and password file <none>
I20260806 22:37:25.703619 20055 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:25.703661 20055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:25.703763 20055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:25.704828 20055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "d7c22fa0120d4004a0cd70dbb7904078"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "41647430cba0cda81777e3853730425f"
server_key_iv: "aebc6ddd544c8169dc04aabc1f0bf242"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.705152 20055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "d7c22fa0120d4004a0cd70dbb7904078"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "41647430cba0cda81777e3853730425f"
server_key_iv: "aebc6ddd544c8169dc04aabc1f0bf242"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.706501 20055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:37:25.707363 20075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.707551 20055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:25.707645 20055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "d7c22fa0120d4004a0cd70dbb7904078"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "41647430cba0cda81777e3853730425f"
server_key_iv: "aebc6ddd544c8169dc04aabc1f0bf242"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.707721 20055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:25.718984 20055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:25.721087 20055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:25.721262 20055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:25.721877 20055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:25.721920 20055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.721943 20055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:25.721956 20055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:25.730195 20055 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:35115
I20260806 22:37:25.730257 20242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:35115 every 8 connection(s)
I20260806 22:37:25.730587 20055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:25.733498 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.722344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:54603 (local address 127.16.235.126:45091)
0806 22:37:25.722421 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.722424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.722719 (+ 295us) server_negotiation.cc:407] Connection header received
0806 22:37:25.723680 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.723681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.723701 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.723756 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:25.724780 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.725688 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.727515 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.727737 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.727804 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.730736 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:25.730738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:25.730738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:25.730747 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:25.732185 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.732590 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.732590 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.732591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.732645 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.732880 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.732881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.732881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.732995 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:25.733047 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.733348 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.733433 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:37:25.733594 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.722311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45091 (local address 127.16.235.66:54603)
0806 22:37:25.722532 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.722594 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.723538 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.723848 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.723853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.724317 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:25.724660 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.724666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.725798 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.725799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.727365 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.727369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.727853 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.727855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.728027 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.728651 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:25.728661 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:25.730592 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:25.732269 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.732276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.732278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.732511 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.732702 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.732703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.732703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.732815 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.733050 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:25.733055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:25.733174 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.733253 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.733393 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260806 22:37:25.734145 20245 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45091
I20260806 22:37:25.734239 20245 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:25.734436 20245 heartbeater.cc:511] Master 127.16.235.126:45091 requested a full tablet report, sending...
I20260806 22:37:25.734956 19756 ts_manager.cc:194] Registered new tserver with Master: d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66:35115)
I20260806 22:37:25.735546 19756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:54603
I20260806 22:37:25.737588 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 20055
I20260806 22:37:25.737671 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:25.737877 17325 external_mini_cluster.cc:1428] Setting key 6b4e5e1ae18ae7823d5dc9af1d1a6875
I20260806 22:37:25.741359 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.735523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:34319 (local address 127.16.235.126:45091)
0806 22:37:25.735566 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.735568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.735657 (+ 89us) server_negotiation.cc:407] Connection header received
0806 22:37:25.735785 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.735786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.735796 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.735839 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:25.736190 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.736672 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.737428 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.737654 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.737717 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.738881 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:25.738883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:25.738883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:25.738891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:25.740160 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.740513 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.740515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.740515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.740577 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.740786 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.740787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.740787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.740936 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:25.741016 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.741188 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.741267 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:37:25.741365 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.735453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45091 (local address 127.16.235.66:34319)
0806 22:37:25.735588 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.735599 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.735686 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.735854 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.735855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.736042 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:25.736100 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.736101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.736772 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.736773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.737343 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.737345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.737772 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.737773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.737837 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.738373 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:25.738378 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:25.738793 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:25.740252 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.740253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.740254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.740437 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.740653 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.740654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.740654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.740723 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.741018 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:25.741020 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:25.741068 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.741149 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.741269 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:25.767431 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45091
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:25.852841 20257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:25.853104 20257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:25.853152 20257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:25.854334 20257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:25.854403 20257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:25.854439 20257 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:37:25.854480 20257 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:37:25.854521 20257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:25.856081 20257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.235.126:45091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:25.856366 20257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:25.856616 20257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:25.861903 20264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:25.861922 20265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:25.861905 20267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:25.862080 20257 server_base.cc:1034] running on GCE node
I20260806 22:37:25.862373 20257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:25.862633 20257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:25.863840 20257 hybrid_clock.cc:648] HybridClock initialized: now 1786055845863820 us; error 43 us; skew 500 ppm
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:25.866317 20257 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:25.866930 20257 webserver.cc:460] Webserver started at http://127.16.235.67:45693/ using document root <none> and password file <none>
I20260806 22:37:25.867242 20257 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:25.867317 20257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:25.867499 20257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:25.868693 20257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "5356bf3eed2e4605b595e3299dc1adb5"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "3ca9ca3a52d67d3c762c589371d27a41"
server_key_iv: "997312ed1c844d2efcc126d88c428680"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.869091 20257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "5356bf3eed2e4605b595e3299dc1adb5"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "3ca9ca3a52d67d3c762c589371d27a41"
server_key_iv: "997312ed1c844d2efcc126d88c428680"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.870487 20257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:37:25.871344 20275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.871517 20257 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:37:25.871621 20257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "5356bf3eed2e4605b595e3299dc1adb5"
format_stamp: "Formatted at 2026-08-06 22:37:25 on dist-test-slave-3kk6"
server_key: "3ca9ca3a52d67d3c762c589371d27a41"
server_key_iv: "997312ed1c844d2efcc126d88c428680"
server_key_version: "encryptionkey@0"
I20260806 22:37:25.871717 20257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:25.888689 20257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:25.891026 20257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:25.891218 20257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:25.891860 20257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:25.891922 20257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:25.891960 20257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:25.891988 20257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:25.901470 20257 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:45809
I20260806 22:37:25.901602 20434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:45809 every 8 connection(s)
I20260806 22:37:25.901921 20257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:25.902631 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.892264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45091 (local address 127.16.235.67:49829)
0806 22:37:25.892487 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.892532 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.893041 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.893326 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.893329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.893630 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:25.893830 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.893835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.894931 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.894932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.895872 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.895875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.896288 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.896289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.896448 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.897019 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:25.897027 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:25.898954 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:25.900877 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.900884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.900886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.901250 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.901515 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.901515 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.901516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.901656 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.901964 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:25.901972 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:25.902139 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.902258 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.902447 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":152,"threads_started":1}
I20260806 22:37:25.902951 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.892303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:49829 (local address 127.16.235.126:45091)
0806 22:37:25.892378 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.892381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.892632 (+ 251us) server_negotiation.cc:407] Connection header received
0806 22:37:25.893146 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.893148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.893162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.893267 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:25.893937 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.894827 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.895968 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.896190 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.896271 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.899072 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:25.899074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:25.899075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:25.899085 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:25.900774 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.901361 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.901363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.901364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.901437 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.901752 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.901754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.901754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.901882 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:25.901960 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.902771 (+ 811us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.902865 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:37:25.903349 20437 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45091
I20260806 22:37:25.903439 20437 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:25.903676 20437 heartbeater.cc:511] Master 127.16.235.126:45091 requested a full tablet report, sending...
I20260806 22:37:25.903702 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 20257
I20260806 22:37:25.903808 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:25.904072 17325 external_mini_cluster.cc:1428] Setting key 1683e01078fc57165c0672b95bf8506b
I20260806 22:37:25.904088 19755 ts_manager.cc:194] Registered new tserver with Master: 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809)
I20260806 22:37:25.904560 19755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:49829
I20260806 22:37:25.905243 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:37:25.911513 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.904570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45091 (local address 127.16.235.67:55745)
0806 22:37:25.904692 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.904703 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.904796 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.904916 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.904916 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.905165 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:25.905263 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.905265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.906198 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.906199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.907076 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.907079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.907427 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.907428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.907514 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.908079 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:25.908084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:25.908385 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:25.910272 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.910274 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.910274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.910526 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.910772 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:25.910773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:25.910773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:25.910848 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:25.911141 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:25.911141 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:25.911203 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.911276 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.911402 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260806 22:37:25.911550 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.904630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:55745 (local address 127.16.235.126:45091)
0806 22:37:25.904669 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.904671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.904784 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:37:25.904856 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.904858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.904868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.904913 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:25.905364 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.906061 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.907161 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.907344 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.907405 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.908473 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:25.908475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:25.908475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:25.908483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:25.910162 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.910597 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.910598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.910599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.910682 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.910916 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.910917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.910917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.911072 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:25.911149 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.911324 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.911413 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:37:25 dist-test-slave-3kk6 krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:25.931531 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.924368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40834 (local address 127.16.235.126:45091)
0806 22:37:25.924441 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.924443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.924455 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:37:25.924491 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.924492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.924501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.924562 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:25.925195 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.925689 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.926443 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.926634 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.926760 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.929001 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:25.929002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:25.929003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:25.929011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:25.930353 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.930725 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.930726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.930727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.930792 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:25.931000 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:25.931001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:25.931001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:25.931119 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:25.931228 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.931337 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.931424 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:37:25.933218 19755 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:25.935274 19755 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:25.942227 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.938045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:36347 (local address 127.0.0.1:35338)
0806 22:37:25.938351 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.938397 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.938481 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.938836 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.938840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.938842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:25.938938 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.938944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.939897 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.939898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.941154 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.941157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.941876 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.941877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.941958 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.941972 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.942075 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.942133 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260806 22:37:25.942237 20458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.938157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35338 (local address 127.16.235.65:36347)
0806 22:37:25.938413 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.938420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.938454 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:37:25.938619 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.938621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.938639 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.938736 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:25.939029 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.939757 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.941248 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.941780 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.941876 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.941943 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.942095 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.942136 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":91,"threads_started":1}
I20260806 22:37:25.942499 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.938294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35115 (local address 127.0.0.1:47548)
0806 22:37:25.938549 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.938558 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.938599 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.938951 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.938952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.938954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:25.938989 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.938991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.940291 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.940292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.941401 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.941404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.942283 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.942284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.942339 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.942343 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.942375 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.942426 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":25,"thread_start_us":99,"threads_started":1}
I20260806 22:37:25.942591 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.938134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.16.235.67:45809)
0806 22:37:25.938332 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.938337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.938520 (+ 183us) server_negotiation.cc:407] Connection header received
0806 22:37:25.938681 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.938684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.938701 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.938766 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:25.939025 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.940158 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.941351 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.942083 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.942160 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.942216 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.942443 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.942486 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260806 22:37:25.942852 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.938566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.16.235.66:35115)
0806 22:37:25.938768 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:25.938773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:25.938783 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:37:25.938835 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:25.938837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:25.938853 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:25.939062 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:25.939124 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.940207 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.941495 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.942192 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.942577 (+ 385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.942673 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:25.942702 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 22:37:25.942741 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":127,"threads_started":1}
I20260806 22:37:25.943369 19953 tablet_service.cc:1467] Processing CreateTablet for tablet fabfdd8734874ca097957bcc5ec9aa71 (DEFAULT_TABLE table=test-table [id=5578a24bd5a2451c911563ccf4b5c6ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:25.943809 19953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fabfdd8734874ca097957bcc5ec9aa71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:25.942804 20459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:25.938047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:45809 (local address 127.0.0.1:35814)
0806 22:37:25.938437 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:25.938451 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:25.938554 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:25.938868 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:25.938869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:25.938872 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:25.938938 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.938940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.940268 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.940269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.941255 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:25.941258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:25.942257 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:25.942259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:25.942346 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:25.942349 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:25.942698 (+ 349us) client_negotiation.cc:240] Negotiation successful
0806 22:37:25.942742 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":34,"thread_start_us":105,"threads_started":1}
I20260806 22:37:25.946942 20461 tablet_bootstrap.cc:492] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: Bootstrap starting.
I20260806 22:37:25.947777 20461 tablet_bootstrap.cc:654] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:25.947718 20341 tablet_service.cc:1467] Processing CreateTablet for tablet fabfdd8734874ca097957bcc5ec9aa71 (DEFAULT_TABLE table=test-table [id=5578a24bd5a2451c911563ccf4b5c6ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:25.947777 20150 tablet_service.cc:1467] Processing CreateTablet for tablet fabfdd8734874ca097957bcc5ec9aa71 (DEFAULT_TABLE table=test-table [id=5578a24bd5a2451c911563ccf4b5c6ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:25.948119 20150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fabfdd8734874ca097957bcc5ec9aa71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:25.948119 20341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fabfdd8734874ca097957bcc5ec9aa71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:25.948143 20461 log.cc:824] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:25.948908 20461 tablet_bootstrap.cc:492] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: No bootstrap required, opened a new log
I20260806 22:37:25.948997 20461 ts_tablet_manager.cc:1397] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:25.950472 20461 raft_consensus.cc:348] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } }
I20260806 22:37:25.950637 20461 raft_consensus.cc:374] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:25.950709 20461 raft_consensus.cc:729] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58170824e82419d9c5b35917150641b, State: Initialized, Role: FOLLOWER
I20260806 22:37:25.950848 20461 consensus_queue.cc:260] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } }
I20260806 22:37:25.951174 20045 heartbeater.cc:503] Master 127.16.235.126:45091 was elected leader, sending a full tablet report...
I20260806 22:37:25.951164 20461 ts_tablet_manager.cc:1428] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:37:25.951647 20464 tablet_bootstrap.cc:492] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: Bootstrap starting.
I20260806 22:37:25.952256 20465 tablet_bootstrap.cc:492] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: Bootstrap starting.
I20260806 22:37:25.952702 20464 tablet_bootstrap.cc:654] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:25.953063 20464 log.cc:824] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:25.953176 20465 tablet_bootstrap.cc:654] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:25.953526 20465 log.cc:824] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:25.953931 20464 tablet_bootstrap.cc:492] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: No bootstrap required, opened a new log
I20260806 22:37:25.953989 20464 ts_tablet_manager.cc:1397] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:37:25.954352 20465 tablet_bootstrap.cc:492] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: No bootstrap required, opened a new log
I20260806 22:37:25.954427 20465 ts_tablet_manager.cc:1397] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:25.956071 20464 raft_consensus.cc:348] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } }
I20260806 22:37:25.956257 20464 raft_consensus.cc:374] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:25.956303 20464 raft_consensus.cc:729] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5356bf3eed2e4605b595e3299dc1adb5, State: Initialized, Role: FOLLOWER
I20260806 22:37:25.956435 20464 consensus_queue.cc:260] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } }
I20260806 22:37:25.956441 20465 raft_consensus.cc:348] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } }
I20260806 22:37:25.956599 20465 raft_consensus.cc:374] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:25.956655 20465 raft_consensus.cc:729] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c22fa0120d4004a0cd70dbb7904078, State: Initialized, Role: FOLLOWER
I20260806 22:37:25.956733 20464 ts_tablet_manager.cc:1428] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:37:25.956789 20437 heartbeater.cc:503] Master 127.16.235.126:45091 was elected leader, sending a full tablet report...
I20260806 22:37:25.956772 20465 consensus_queue.cc:260] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } }
I20260806 22:37:25.957103 20245 heartbeater.cc:503] Master 127.16.235.126:45091 was elected leader, sending a full tablet report...
I20260806 22:37:25.957091 20465 ts_tablet_manager.cc:1428] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 22:37:25.981230 20246 tablet.cc:2369] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:26.064435 20046 tablet.cc:2369] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:26.152585 20438 tablet.cc:2369] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:26.199645 20463 raft_consensus.cc:482] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:26.199869 20463 raft_consensus.cc:504] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } }
I20260806 22:37:26.200312 20463 leader_election.cc:288] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809), d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66:35115)
I20260806 22:37:26.203862 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.201023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:36571 (local address 127.16.235.66:35115)
0806 22:37:26.201078 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.201080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.201089 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:26.201121 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.201122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.201131 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.201189 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:26.201372 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.201888 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.203055 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.203443 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.203503 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.203511 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.203736 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.203793 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:37:26.203857 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.200849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35115 (local address 127.16.235.65:36571)
0806 22:37:26.201000 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.201012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.201069 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.201202 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.201203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.201205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:26.201293 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.201295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.201982 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.201982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.202914 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.202918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.203538 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.203539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.203620 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.203624 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.203684 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.203727 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":15,"thread_start_us":39,"threads_started":1}
I20260806 22:37:26.204043 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.200615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:45809 (local address 127.16.235.65:40545)
0806 22:37:26.200762 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.200782 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.200848 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.201039 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.201042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.201045 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:26.201135 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.201138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.201912 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.201913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.202980 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.202984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.203640 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.203641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.203729 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.203734 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.203793 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.203834 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":56,"thread_start_us":102,"threads_started":1}
I20260806 22:37:26.204272 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.200760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:40545 (local address 127.16.235.67:45809)
0806 22:37:26.200853 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.200855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.200876 (+ 21us) server_negotiation.cc:407] Connection header received
0806 22:37:26.200918 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.200919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.200931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.201019 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:26.201258 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.201811 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.203081 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.203457 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.203604 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.203614 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.203895 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.203942 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:37:26.204293 20171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fabfdd8734874ca097957bcc5ec9aa71" candidate_uuid: "e58170824e82419d9c5b35917150641b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c22fa0120d4004a0cd70dbb7904078" is_pre_election: true
I20260806 22:37:26.204504 20171 raft_consensus.cc:2454] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e58170824e82419d9c5b35917150641b in term 0.
I20260806 22:37:26.204552 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fabfdd8734874ca097957bcc5ec9aa71" candidate_uuid: "e58170824e82419d9c5b35917150641b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5356bf3eed2e4605b595e3299dc1adb5" is_pre_election: true
I20260806 22:37:26.204730 20365 raft_consensus.cc:2454] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e58170824e82419d9c5b35917150641b in term 0.
I20260806 22:37:26.204834 19885 leader_election.cc:302] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7c22fa0120d4004a0cd70dbb7904078, e58170824e82419d9c5b35917150641b; no voters:
I20260806 22:37:26.205044 20463 raft_consensus.cc:2781] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:26.205116 20463 raft_consensus.cc:482] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:26.205137 20463 raft_consensus.cc:3037] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:26.205940 20463 raft_consensus.cc:504] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } }
I20260806 22:37:26.206092 20463 leader_election.cc:288] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [CANDIDATE]: Term 1 election: Requested vote from peers 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809), d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66:35115)
I20260806 22:37:26.206313 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fabfdd8734874ca097957bcc5ec9aa71" candidate_uuid: "e58170824e82419d9c5b35917150641b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5356bf3eed2e4605b595e3299dc1adb5"
I20260806 22:37:26.206350 20171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fabfdd8734874ca097957bcc5ec9aa71" candidate_uuid: "e58170824e82419d9c5b35917150641b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c22fa0120d4004a0cd70dbb7904078"
I20260806 22:37:26.206394 20365 raft_consensus.cc:3037] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:26.206447 20171 raft_consensus.cc:3037] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:26.207307 20171 raft_consensus.cc:2454] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e58170824e82419d9c5b35917150641b in term 1.
I20260806 22:37:26.207300 20365 raft_consensus.cc:2454] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e58170824e82419d9c5b35917150641b in term 1.
I20260806 22:37:26.207460 19885 leader_election.cc:302] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5356bf3eed2e4605b595e3299dc1adb5, e58170824e82419d9c5b35917150641b; no voters:
I20260806 22:37:26.207571 20463 raft_consensus.cc:2781] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:26.207736 20463 raft_consensus.cc:686] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 1 LEADER]: Becoming Leader. State: Replica: e58170824e82419d9c5b35917150641b, State: Running, Role: LEADER
I20260806 22:37:26.207845 20463 consensus_queue.cc:237] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } }
I20260806 22:37:26.208688 19755 catalog_manager.cc:5495] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b reported cstate change: term changed from 0 to 1, leader changed from <none> to e58170824e82419d9c5b35917150641b (127.16.235.65). New cstate: current_term: 1 leader_uuid: "e58170824e82419d9c5b35917150641b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:37:26 dist-test-slave-3kk6 krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055845, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:26.254911 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.247551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.16.235.126:45091)
0806 22:37:26.247642 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.247644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.247769 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:37:26.247870 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.247871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.247882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.247958 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:26.248453 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.248963 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.249680 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.249871 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.249959 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.252190 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:26.252191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:26.252192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:26.252209 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:26.253633 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:26.254050 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:26.254052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:26.254052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:26.254113 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:26.254318 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:26.254319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:26.254320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:26.254438 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:26.254518 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.254708 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.254799 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:37:26.254983 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.247458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45091 (local address 127.0.0.1:40846)
0806 22:37:26.247665 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.247686 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.247792 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.247965 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.247966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.248309 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:26.248358 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.248361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.249055 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.249057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.249588 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.249589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.249960 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.249961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.250024 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.250565 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:26.250570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:26.252074 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:26.253753 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:26.253761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:26.253764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:26.253970 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:26.254190 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:26.254191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:26.254191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:26.254253 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:26.254508 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:26.254517 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:26.254565 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.254652 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.254856 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":61,"thread_start_us":117,"threads_started":1}
I20260806 22:37:26.256968 19755 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:40846:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:37:26.257220 19755 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:26.258960 20150 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1450ab7d7a4f51a13194848a1ff28d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e129ed8da50f4b0380b3b053ac2a83f4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:37:26.258952 19953 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1450ab7d7a4f51a13194848a1ff28d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e129ed8da50f4b0380b3b053ac2a83f4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:37:26.259153 19953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1450ab7d7a4f51a13194848a1ff28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:26.259153 20150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1450ab7d7a4f51a13194848a1ff28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:26.260797 20461 tablet_bootstrap.cc:492] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: Bootstrap starting.
I20260806 22:37:26.260795 20465 tablet_bootstrap.cc:492] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: Bootstrap starting.
I20260806 22:37:26.260946 20341 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1450ab7d7a4f51a13194848a1ff28d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e129ed8da50f4b0380b3b053ac2a83f4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:37:26.261142 20341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1450ab7d7a4f51a13194848a1ff28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:26.261339 20465 tablet_bootstrap.cc:654] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:26.261340 20461 tablet_bootstrap.cc:654] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:26.262137 20465 tablet_bootstrap.cc:492] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: No bootstrap required, opened a new log
I20260806 22:37:26.262212 20465 ts_tablet_manager.cc:1397] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:26.262408 20465 raft_consensus.cc:348] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.262504 20465 raft_consensus.cc:374] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:26.262529 20465 raft_consensus.cc:729] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c22fa0120d4004a0cd70dbb7904078, State: Initialized, Role: FOLLOWER
I20260806 22:37:26.262578 20465 consensus_queue.cc:260] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.262768 20465 ts_tablet_manager.cc:1428] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:26.262758 20464 tablet_bootstrap.cc:492] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: Bootstrap starting.
I20260806 22:37:26.262830 20472 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.262919 20472 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:26.263370 20464 tablet_bootstrap.cc:654] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:26.263473 20461 tablet_bootstrap.cc:492] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: No bootstrap required, opened a new log
I20260806 22:37:26.263523 20461 ts_tablet_manager.cc:1397] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:37:26.263938 20461 raft_consensus.cc:348] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.264032 20461 raft_consensus.cc:374] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:26.264056 20461 raft_consensus.cc:729] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58170824e82419d9c5b35917150641b, State: Initialized, Role: FOLLOWER
I20260806 22:37:26.264110 20461 consensus_queue.cc:260] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.264200 20464 tablet_bootstrap.cc:492] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: No bootstrap required, opened a new log
I20260806 22:37:26.264221 20461 ts_tablet_manager.cc:1428] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:26.264250 20464 ts_tablet_manager.cc:1397] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:26.264286 20463 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.264375 20463 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:26.264410 20464 raft_consensus.cc:348] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.264498 20464 raft_consensus.cc:374] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:26.264520 20464 raft_consensus.cc:729] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5356bf3eed2e4605b595e3299dc1adb5, State: Initialized, Role: FOLLOWER
I20260806 22:37:26.264564 20464 consensus_queue.cc:260] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.264675 20464 ts_tablet_manager.cc:1428] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:26.264735 20470 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.264806 20470 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:26.400502 20472 raft_consensus.cc:482] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:26.400640 20472 raft_consensus.cc:504] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.401095 20472 leader_election.cc:288] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e58170824e82419d9c5b35917150641b (127.16.235.65:36347), 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809)
I20260806 22:37:26.404592 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.401496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:49547 (local address 127.16.235.67:45809)
0806 22:37:26.401546 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.401548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.401641 (+ 93us) server_negotiation.cc:407] Connection header received
0806 22:37:26.401770 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.401771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.401781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.401860 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:26.402024 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.402499 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.403694 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.404153 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.404268 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.404278 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.404479 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.404518 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:37:26.404616 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.401374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:45809 (local address 127.16.235.66:49547)
0806 22:37:26.401548 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.401560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.401656 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.401867 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.401868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.401870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:26.401950 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.401951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.402592 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.402593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.403577 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.403580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.404340 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.404342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.404412 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.404417 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.404472 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.404517 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
I20260806 22:37:26.404946 20458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.401409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:60521 (local address 127.16.235.65:36347)
0806 22:37:26.401484 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.401486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.401562 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:37:26.401646 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.401647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.401659 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.401723 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:26.401909 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.402400 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.403722 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.404191 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.404273 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.404284 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.404826 (+ 542us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.404878 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:37:26.404958 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.401320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:36347 (local address 127.16.235.66:60521)
0806 22:37:26.401483 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.401497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.401572 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.401745 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.401746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.401748 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:26.401830 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.401832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.402493 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.402495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.403629 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.403632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.404652 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.404654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.404726 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.404732 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.404810 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.404858 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":72,"thread_start_us":96,"threads_started":1}
I20260806 22:37:26.405000 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1450ab7d7a4f51a13194848a1ff28d" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5356bf3eed2e4605b595e3299dc1adb5" is_pre_election: true
I20260806 22:37:26.405102 20365 raft_consensus.cc:2454] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 0.
I20260806 22:37:26.405308 19981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1450ab7d7a4f51a13194848a1ff28d" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58170824e82419d9c5b35917150641b" is_pre_election: true
I20260806 22:37:26.405339 20078 leader_election.cc:302] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5356bf3eed2e4605b595e3299dc1adb5, d7c22fa0120d4004a0cd70dbb7904078; no voters:
I20260806 22:37:26.405447 19981 raft_consensus.cc:2454] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 0.
I20260806 22:37:26.405470 20472 raft_consensus.cc:2781] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:26.405517 20472 raft_consensus.cc:482] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:26.405541 20472 raft_consensus.cc:3037] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:26.406324 20472 raft_consensus.cc:504] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.406483 20472 leader_election.cc:288] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 election: Requested vote from peers e58170824e82419d9c5b35917150641b (127.16.235.65:36347), 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809)
I20260806 22:37:26.406631 19981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1450ab7d7a4f51a13194848a1ff28d" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58170824e82419d9c5b35917150641b"
I20260806 22:37:26.406759 19981 raft_consensus.cc:3037] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:26.406734 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1450ab7d7a4f51a13194848a1ff28d" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5356bf3eed2e4605b595e3299dc1adb5"
I20260806 22:37:26.406834 20365 raft_consensus.cc:3037] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:26.407476 20365 raft_consensus.cc:2454] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 1.
I20260806 22:37:26.407554 19981 raft_consensus.cc:2454] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 1.
I20260806 22:37:26.407641 20078 leader_election.cc:302] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5356bf3eed2e4605b595e3299dc1adb5, d7c22fa0120d4004a0cd70dbb7904078; no voters:
I20260806 22:37:26.407742 20472 raft_consensus.cc:2781] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:26.407859 20472 raft_consensus.cc:686] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [term 1 LEADER]: Becoming Leader. State: Replica: d7c22fa0120d4004a0cd70dbb7904078, State: Running, Role: LEADER
I20260806 22:37:26.407963 20472 consensus_queue.cc:237] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.408443 20493 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: TxnStatusTablet state changed. Reason: New leader d7c22fa0120d4004a0cd70dbb7904078. Latest consensus state: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.408571 20493 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:37:26.408794 20495 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:37:26.408927 20495 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:37:26.408864 19756 catalog_manager.cc:5495] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66). New cstate: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:26.446201 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.443166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35115 (local address 127.0.0.1:47560)
0806 22:37:26.443260 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.443272 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.443331 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.443542 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.443544 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.443545 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:26.443627 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.443630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.444432 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.444434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.445152 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.445155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.445589 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.445591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.445670 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.445696 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.446012 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.446023 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.446074 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.446130 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:37:26.446197 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.443206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.16.235.66:35115)
0806 22:37:26.443252 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.443254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.443308 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:37:26.443413 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.443414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.443435 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.443516 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:26.443708 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.444338 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.445240 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.445495 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.445586 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.445818 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.445941 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.446037 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.446081 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.446107 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:37:26.448343 19981 raft_consensus.cc:1261] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Refusing update from remote peer d7c22fa0120d4004a0cd70dbb7904078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:26.448364 20365 raft_consensus.cc:1261] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Refusing update from remote peer d7c22fa0120d4004a0cd70dbb7904078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:26.448709 20493 consensus_queue.cc:1035] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:26.448863 20472 consensus_queue.cc:1035] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:26.450145 20470 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: TxnStatusTablet state changed. Reason: New leader d7c22fa0120d4004a0cd70dbb7904078. Latest consensus state: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.450253 20470 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:26.450249 20463 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: TxnStatusTablet state changed. Reason: New leader d7c22fa0120d4004a0cd70dbb7904078. Latest consensus state: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.450372 20463 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:26.451260 20498 mvcc.cc:205] Tried to move back new op lower bound from 7315684747049299968 to 7315684746887520256. Current Snapshot: MvccSnapshot[applied={T|T < 7315684747049299968}]
I20260806 22:37:26.451380 20472 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.451460 20472 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:37:26.451740 20500 mvcc.cc:205] Tried to move back new op lower bound from 7315684747049299968 to 7315684746887520256. Current Snapshot: MvccSnapshot[applied={T|T < 7315684747049299968}]
I20260806 22:37:26.451906 20463 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.451994 20463 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P e58170824e82419d9c5b35917150641b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:26.452170 20472 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.452253 20472 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P d7c22fa0120d4004a0cd70dbb7904078: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:37:26.452517 20470 tablet_replica.cc:439] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } } }
I20260806 22:37:26.452600 20470 tablet_replica.cc:442] T 9d1450ab7d7a4f51a13194848a1ff28d P 5356bf3eed2e4605b595e3299dc1adb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:26.452942 20499 mvcc.cc:205] Tried to move back new op lower bound from 7315684747049299968 to 7315684746887520256. Current Snapshot: MvccSnapshot[applied={T|T < 7315684747049299968}]
I20260806 22:37:26.457528 20458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.454722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35344 (local address 127.16.235.65:36347)
0806 22:37:26.454777 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.454780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.454893 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:37:26.454974 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.454974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.454990 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.455049 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:26.455191 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.455655 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.456807 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.456985 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.457036 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.457209 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.457304 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.457361 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.457433 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.457451 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:37:26.462580 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.460144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:38637 (local address 127.16.235.66:35115)
0806 22:37:26.460191 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.460194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.460314 (+ 120us) server_negotiation.cc:407] Connection header received
0806 22:37:26.460385 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.460387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.460397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.460451 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:26.460582 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.461066 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.461829 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.462038 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.462116 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.462310 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.462346 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.462416 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.462479 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.462501 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:37:26.462613 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.460069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35115 (local address 127.16.235.65:38637)
0806 22:37:26.460235 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.460245 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.460332 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.460455 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.460455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.460457 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:26.460520 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.460521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.461161 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.461162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.461741 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.461743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.462114 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.462115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.462185 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.462198 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.462418 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.462421 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.462474 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.462516 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260806 22:37:26.465185 20365 raft_consensus.cc:1261] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Refusing update from remote peer e58170824e82419d9c5b35917150641b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:26.465210 20171 raft_consensus.cc:1261] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Refusing update from remote peer e58170824e82419d9c5b35917150641b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:26.465451 20463 consensus_queue.cc:1035] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:26.465605 20476 consensus_queue.cc:1035] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:26.475440 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.472366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:36347 (local address 127.16.235.66:40931)
0806 22:37:26.472539 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.472552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.472619 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.472801 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.472802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.472803 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:26.472856 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.472858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.473479 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.473480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.474337 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.474341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.474996 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.474997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.475062 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.475079 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.475244 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.475246 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.475301 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.475351 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260806 22:37:26.475459 20458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.472429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:40931 (local address 127.16.235.65:36347)
0806 22:37:26.472527 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.472530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.472607 (+ 77us) server_negotiation.cc:407] Connection header received
0806 22:37:26.472680 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.472681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.472692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.472743 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:26.472911 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.473393 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.474598 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.474918 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.474974 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.475175 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.475204 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.475239 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.475302 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.475323 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:37:26.485198 19756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:26.485343 19756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:26.486238 19756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a: Sending DeleteTablet for 3 replicas of tablet fabfdd8734874ca097957bcc5ec9aa71
I20260806 22:37:26.486580 20341 tablet_service.cc:1514] Processing DeleteTablet for tablet fabfdd8734874ca097957bcc5ec9aa71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:26 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35814
I20260806 22:37:26.486606 20150 tablet_service.cc:1514] Processing DeleteTablet for tablet fabfdd8734874ca097957bcc5ec9aa71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:26 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47548
I20260806 22:37:26.487011 19953 tablet_service.cc:1514] Processing DeleteTablet for tablet fabfdd8734874ca097957bcc5ec9aa71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:26 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35338
I20260806 22:37:26.487293 20535 tablet_replica.cc:330] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: stopping tablet replica
I20260806 22:37:26.487318 19756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:26.487478 19756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:26.487627 20535 raft_consensus.cc:2229] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:26.487802 20535 raft_consensus.cc:2258] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:26.488255 20535 ts_tablet_manager.cc:1905] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:26.489005 20341 tablet_service.cc:1467] Processing CreateTablet for tablet bc96609b5d08467e9ff0c0cc3408e027 (DEFAULT_TABLE table=test-table [id=e2b70b5ba1464e6ba8cf877d4d28681b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:26.489168 20341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc96609b5d08467e9ff0c0cc3408e027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:26.490391 20535 ts_tablet_manager.cc:1918] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:26.490545 20535 log.cc:1196] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/fabfdd8734874ca097957bcc5ec9aa71
I20260806 22:37:26.490721 20464 tablet_bootstrap.cc:492] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: Bootstrap starting.
I20260806 22:37:26.491009 20535 ts_tablet_manager.cc:1939] T fabfdd8734874ca097957bcc5ec9aa71 P e58170824e82419d9c5b35917150641b: Deleting consensus metadata
I20260806 22:37:26.491142 20464 tablet_bootstrap.cc:654] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:26.491865 20150 tablet_service.cc:1467] Processing CreateTablet for tablet bc96609b5d08467e9ff0c0cc3408e027 (DEFAULT_TABLE table=test-table [id=e2b70b5ba1464e6ba8cf877d4d28681b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:26.492003 20150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc96609b5d08467e9ff0c0cc3408e027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:26.493422 20465 tablet_bootstrap.cc:492] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: Bootstrap starting.
I20260806 22:37:26.493604 19737 catalog_manager.cc:4841] TS e58170824e82419d9c5b35917150641b (127.16.235.65:36347): tablet fabfdd8734874ca097957bcc5ec9aa71 (table test-table [id=5578a24bd5a2451c911563ccf4b5c6ca]) successfully deleted
I20260806 22:37:26.493844 20465 tablet_bootstrap.cc:654] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:26.493973 19953 tablet_service.cc:1467] Processing CreateTablet for tablet bc96609b5d08467e9ff0c0cc3408e027 (DEFAULT_TABLE table=test-table [id=e2b70b5ba1464e6ba8cf877d4d28681b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:26.494084 19953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc96609b5d08467e9ff0c0cc3408e027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:26.495457 20461 tablet_bootstrap.cc:492] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: Bootstrap starting.
I20260806 22:37:26.495898 20461 tablet_bootstrap.cc:654] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:26.496161 20464 tablet_bootstrap.cc:492] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: No bootstrap required, opened a new log
I20260806 22:37:26.496212 20464 ts_tablet_manager.cc:1397] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260806 22:37:26.496381 20464 raft_consensus.cc:348] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.496472 20464 raft_consensus.cc:374] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:26.496497 20464 raft_consensus.cc:729] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5356bf3eed2e4605b595e3299dc1adb5, State: Initialized, Role: FOLLOWER
I20260806 22:37:26.496548 20464 consensus_queue.cc:260] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.496663 20464 ts_tablet_manager.cc:1428] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:37:26.496961 20532 tablet_replica.cc:330] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: stopping tablet replica
I20260806 22:37:26.497429 20532 raft_consensus.cc:2229] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:26.497514 20532 raft_consensus.cc:2258] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:26.497677 20530 tablet_replica.cc:330] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: stopping tablet replica
I20260806 22:37:26.497807 20530 raft_consensus.cc:2229] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:26.497876 20530 raft_consensus.cc:2258] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:26.498216 20532 ts_tablet_manager.cc:1905] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:26.499070 20465 tablet_bootstrap.cc:492] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: No bootstrap required, opened a new log
I20260806 22:37:26.499121 20465 ts_tablet_manager.cc:1397] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260806 22:37:26.499259 20465 raft_consensus.cc:348] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.499334 20465 raft_consensus.cc:374] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:26.499354 20465 raft_consensus.cc:729] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c22fa0120d4004a0cd70dbb7904078, State: Initialized, Role: FOLLOWER
I20260806 22:37:26.499395 20465 consensus_queue.cc:260] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.499472 20465 ts_tablet_manager.cc:1428] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:26.500206 20461 tablet_bootstrap.cc:492] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: No bootstrap required, opened a new log
I20260806 22:37:26.500257 20461 ts_tablet_manager.cc:1397] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 22:37:26.500317 20530 ts_tablet_manager.cc:1905] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:26.500411 20461 raft_consensus.cc:348] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.500478 20461 raft_consensus.cc:374] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:26.500499 20461 raft_consensus.cc:729] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58170824e82419d9c5b35917150641b, State: Initialized, Role: FOLLOWER
I20260806 22:37:26.500538 20461 consensus_queue.cc:260] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.500681 20461 ts_tablet_manager.cc:1428] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:26.501139 20532 ts_tablet_manager.cc:1918] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:26.501219 20532 log.cc:1196] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/fabfdd8734874ca097957bcc5ec9aa71
I20260806 22:37:26.501521 20532 ts_tablet_manager.cc:1939] T fabfdd8734874ca097957bcc5ec9aa71 P d7c22fa0120d4004a0cd70dbb7904078: Deleting consensus metadata
I20260806 22:37:26.501995 19735 catalog_manager.cc:4841] TS d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66:35115): tablet fabfdd8734874ca097957bcc5ec9aa71 (table test-table [id=5578a24bd5a2451c911563ccf4b5c6ca]) successfully deleted
I20260806 22:37:26.503515 20530 ts_tablet_manager.cc:1918] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:26.503597 20530 log.cc:1196] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/fabfdd8734874ca097957bcc5ec9aa71
I20260806 22:37:26.503898 20530 ts_tablet_manager.cc:1939] T fabfdd8734874ca097957bcc5ec9aa71 P 5356bf3eed2e4605b595e3299dc1adb5: Deleting consensus metadata
I20260806 22:37:26.504448 19735 catalog_manager.cc:4841] TS 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809): tablet fabfdd8734874ca097957bcc5ec9aa71 (table test-table [id=5578a24bd5a2451c911563ccf4b5c6ca]) successfully deleted
I20260806 22:37:26.563143 20472 raft_consensus.cc:482] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:26.563267 20472 raft_consensus.cc:504] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.563448 20472 leader_election.cc:288] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e58170824e82419d9c5b35917150641b (127.16.235.65:36347), 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809)
I20260806 22:37:26.563791 19981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc96609b5d08467e9ff0c0cc3408e027" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58170824e82419d9c5b35917150641b" is_pre_election: true
I20260806 22:37:26.563819 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc96609b5d08467e9ff0c0cc3408e027" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5356bf3eed2e4605b595e3299dc1adb5" is_pre_election: true
I20260806 22:37:26.563896 19981 raft_consensus.cc:2454] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 0.
I20260806 22:37:26.563915 20365 raft_consensus.cc:2454] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 0.
I20260806 22:37:26.564076 20078 leader_election.cc:302] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5356bf3eed2e4605b595e3299dc1adb5, d7c22fa0120d4004a0cd70dbb7904078; no voters:
I20260806 22:37:26.564162 20472 raft_consensus.cc:2781] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:26.564195 20472 raft_consensus.cc:482] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:26.564214 20472 raft_consensus.cc:3037] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:26.565335 20472 raft_consensus.cc:504] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.565479 20472 leader_election.cc:288] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 election: Requested vote from peers e58170824e82419d9c5b35917150641b (127.16.235.65:36347), 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809)
I20260806 22:37:26.565627 19981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc96609b5d08467e9ff0c0cc3408e027" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58170824e82419d9c5b35917150641b"
I20260806 22:37:26.565661 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc96609b5d08467e9ff0c0cc3408e027" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5356bf3eed2e4605b595e3299dc1adb5"
I20260806 22:37:26.565714 19981 raft_consensus.cc:3037] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:26.565724 20365 raft_consensus.cc:3037] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:26.566468 20365 raft_consensus.cc:2454] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 1.
I20260806 22:37:26.566491 19981 raft_consensus.cc:2454] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 1.
I20260806 22:37:26.566638 20078 leader_election.cc:302] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5356bf3eed2e4605b595e3299dc1adb5, d7c22fa0120d4004a0cd70dbb7904078; no voters:
I20260806 22:37:26.566771 20472 raft_consensus.cc:2781] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:26.566819 20472 raft_consensus.cc:686] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 1 LEADER]: Becoming Leader. State: Replica: d7c22fa0120d4004a0cd70dbb7904078, State: Running, Role: LEADER
I20260806 22:37:26.566859 20472 consensus_queue.cc:237] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:26.567535 19755 catalog_manager.cc:5495] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66). New cstate: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:26.577400 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.575105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47574 (local address 127.16.235.66:35115)
0806 22:37:26.575156 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.575158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.575168 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:37:26.575239 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.575239 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.575254 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.575300 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:26.575430 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.575928 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.576676 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.576878 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.576966 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.577123 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.577161 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.577223 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.577293 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.577314 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:37:26.582291 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.579738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35115 (local address 127.16.235.66:42373)
0806 22:37:26.579813 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:26.579826 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:26.579872 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:26.580001 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:26.580001 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:26.580002 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:26.580042 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.580043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.580883 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.580883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.581410 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:26.581411 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.581858 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.581858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:26.581923 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.581932 (+ 9us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.582100 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.582101 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:37:26.582138 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 22:37:26.582191 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":2}
I20260806 22:37:26.582496 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:26.579812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:42373 (local address 127.16.235.66:35115)
0806 22:37:26.579850 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:26.579852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:26.579861 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:26.579918 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:26.579918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:26.579928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:26.579999 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:26.580099 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.580800 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.581504 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:26.581767 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:26.581854 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:26.582012 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:26.582042 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:26.582081 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:26.582416 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 22:37:26.582435 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":20}
I20260806 22:37:26.585659 20365 raft_consensus.cc:1261] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Refusing update from remote peer d7c22fa0120d4004a0cd70dbb7904078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:26.585844 19981 raft_consensus.cc:1261] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Refusing update from remote peer d7c22fa0120d4004a0cd70dbb7904078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:26.585966 20507 consensus_queue.cc:1035] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:26.586093 20493 consensus_queue.cc:1035] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:37:26.591588 20498 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:37:28.115629 19756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:28.115758 19756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:28.116680 19756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a: Sending DeleteTablet for 3 replicas of tablet bc96609b5d08467e9ff0c0cc3408e027
I20260806 22:37:28.117008 19953 tablet_service.cc:1514] Processing DeleteTablet for tablet bc96609b5d08467e9ff0c0cc3408e027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:28 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35338
I20260806 22:37:28.117034 20150 tablet_service.cc:1514] Processing DeleteTablet for tablet bc96609b5d08467e9ff0c0cc3408e027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:28 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47548
I20260806 22:37:28.117252 20341 tablet_service.cc:1514] Processing DeleteTablet for tablet bc96609b5d08467e9ff0c0cc3408e027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:28 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35814
I20260806 22:37:28.117475 20576 tablet_replica.cc:330] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: stopping tablet replica
I20260806 22:37:28.117529 20576 raft_consensus.cc:2229] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:28.117559 20576 raft_consensus.cc:2258] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:28.117635 20574 tablet_replica.cc:330] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: stopping tablet replica
I20260806 22:37:28.117697 20574 raft_consensus.cc:2229] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:28.117782 20574 raft_consensus.cc:2258] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:28.117825 20576 ts_tablet_manager.cc:1905] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:28.117892 20572 tablet_replica.cc:330] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: stopping tablet replica
I20260806 22:37:28.117952 20572 raft_consensus.cc:2229] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:28.117990 20572 raft_consensus.cc:2258] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:28.118248 20572 ts_tablet_manager.cc:1905] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:28.119678 20576 ts_tablet_manager.cc:1918] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:37:28.119768 20576 log.cc:1196] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/bc96609b5d08467e9ff0c0cc3408e027
I20260806 22:37:28.120038 20576 ts_tablet_manager.cc:1939] T bc96609b5d08467e9ff0c0cc3408e027 P 5356bf3eed2e4605b595e3299dc1adb5: Deleting consensus metadata
I20260806 22:37:28.120384 19735 catalog_manager.cc:4841] TS 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809): tablet bc96609b5d08467e9ff0c0cc3408e027 (table test-table [id=e2b70b5ba1464e6ba8cf877d4d28681b]) successfully deleted
I20260806 22:37:28.120479 20572 ts_tablet_manager.cc:1918] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:37:28.120540 20572 log.cc:1196] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/bc96609b5d08467e9ff0c0cc3408e027
I20260806 22:37:28.120775 20574 ts_tablet_manager.cc:1905] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:28.122040 20572 ts_tablet_manager.cc:1939] T bc96609b5d08467e9ff0c0cc3408e027 P e58170824e82419d9c5b35917150641b: Deleting consensus metadata
I20260806 22:37:28.122033 20574 ts_tablet_manager.cc:1918] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:37:28.122124 20574 log.cc:1196] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/bc96609b5d08467e9ff0c0cc3408e027
I20260806 22:37:28.122470 20574 ts_tablet_manager.cc:1939] T bc96609b5d08467e9ff0c0cc3408e027 P d7c22fa0120d4004a0cd70dbb7904078: Deleting consensus metadata
I20260806 22:37:28.122476 19737 catalog_manager.cc:4841] TS e58170824e82419d9c5b35917150641b (127.16.235.65:36347): tablet bc96609b5d08467e9ff0c0cc3408e027 (table test-table [id=e2b70b5ba1464e6ba8cf877d4d28681b]) successfully deleted
I20260806 22:37:28.122869 19735 catalog_manager.cc:4841] TS d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66:35115): tablet bc96609b5d08467e9ff0c0cc3408e027 (table test-table [id=e2b70b5ba1464e6ba8cf877d4d28681b]) successfully deleted
Aug 06 22:37:28 dist-test-slave-3kk6 krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055848, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:37:28 dist-test-slave-3kk6 krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055848, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:28.144533 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.136745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34334 (local address 127.16.235.126:45091)
0806 22:37:28.136937 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:28.136939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:28.137014 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:37:28.137124 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:28.137125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:28.137139 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:28.137220 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:28.137823 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.138380 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.139250 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.139468 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.139541 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.141832 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:28.141833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:28.141834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:28.141841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:28.143258 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:28.143664 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:28.143666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:28.143666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:28.143733 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:28.143951 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:28.143952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:28.143953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:28.144074 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:28.144187 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.144293 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.144386 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
I20260806 22:37:28.146095 19756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:28.146284 19756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:28.147804 20150 tablet_service.cc:1467] Processing CreateTablet for tablet 84ac3a2ee00342a2ad1c0a57afff1dac (DEFAULT_TABLE table=test-table [id=f8b5061a17ab4e56ba8c965291cdb7c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:28.147858 19953 tablet_service.cc:1467] Processing CreateTablet for tablet 84ac3a2ee00342a2ad1c0a57afff1dac (DEFAULT_TABLE table=test-table [id=f8b5061a17ab4e56ba8c965291cdb7c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:28.147858 20341 tablet_service.cc:1467] Processing CreateTablet for tablet 84ac3a2ee00342a2ad1c0a57afff1dac (DEFAULT_TABLE table=test-table [id=f8b5061a17ab4e56ba8c965291cdb7c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:28.147985 20150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84ac3a2ee00342a2ad1c0a57afff1dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:28.147998 20341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84ac3a2ee00342a2ad1c0a57afff1dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:28.148000 19953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84ac3a2ee00342a2ad1c0a57afff1dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:28.149168 20590 tablet_bootstrap.cc:492] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: Bootstrap starting.
I20260806 22:37:28.149497 20591 tablet_bootstrap.cc:492] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: Bootstrap starting.
I20260806 22:37:28.149643 20590 tablet_bootstrap.cc:654] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:28.149904 20591 tablet_bootstrap.cc:654] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:28.150213 20592 tablet_bootstrap.cc:492] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: Bootstrap starting.
I20260806 22:37:28.150457 20590 tablet_bootstrap.cc:492] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: No bootstrap required, opened a new log
I20260806 22:37:28.150504 20590 ts_tablet_manager.cc:1397] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:28.150625 20592 tablet_bootstrap.cc:654] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:28.150604 20590 raft_consensus.cc:348] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.150724 20590 raft_consensus.cc:374] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:28.150764 20590 raft_consensus.cc:729] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c22fa0120d4004a0cd70dbb7904078, State: Initialized, Role: FOLLOWER
I20260806 22:37:28.150820 20590 consensus_queue.cc:260] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.150929 20590 ts_tablet_manager.cc:1428] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:28.151328 20591 tablet_bootstrap.cc:492] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: No bootstrap required, opened a new log
I20260806 22:37:28.151412 20592 tablet_bootstrap.cc:492] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: No bootstrap required, opened a new log
I20260806 22:37:28.151465 20592 ts_tablet_manager.cc:1397] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:28.151386 20591 ts_tablet_manager.cc:1397] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:28.151618 20592 raft_consensus.cc:348] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.151744 20592 raft_consensus.cc:374] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:28.151769 20592 raft_consensus.cc:729] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58170824e82419d9c5b35917150641b, State: Initialized, Role: FOLLOWER
I20260806 22:37:28.151789 20591 raft_consensus.cc:348] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.151840 20592 consensus_queue.cc:260] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.151882 20591 raft_consensus.cc:374] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:28.151907 20591 raft_consensus.cc:729] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5356bf3eed2e4605b595e3299dc1adb5, State: Initialized, Role: FOLLOWER
I20260806 22:37:28.151968 20591 consensus_queue.cc:260] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.152056 20592 ts_tablet_manager.cc:1428] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:28.152148 20591 ts_tablet_manager.cc:1428] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:28.337760 20599 raft_consensus.cc:482] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:28.337882 20599 raft_consensus.cc:504] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.338318 20599 leader_election.cc:288] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e58170824e82419d9c5b35917150641b (127.16.235.65:36347), d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66:35115)
I20260806 22:37:28.342262 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.338624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:38401 (local address 127.16.235.65:36347)
0806 22:37:28.339091 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:28.339094 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:28.339102 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:28.339131 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:28.339132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:28.339143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:28.339210 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:28.339443 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.339977 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.341408 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.341905 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.341983 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.341992 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.342122 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.342163 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":153,"threads_started":1}
I20260806 22:37:28.342226 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.338535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:36347 (local address 127.16.235.67:38401)
0806 22:37:28.338856 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.338871 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:28.338969 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:28.339227 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:28.339228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:28.339230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:28.339342 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:28.339345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.340083 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.340084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:28.341302 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:28.341305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.341989 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.341990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:28.342044 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.342050 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:37:28.342095 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:37:28.342132 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":24,"thread_start_us":236,"threads_started":1}
I20260806 22:37:28.342571 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.338574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35115 (local address 127.16.235.67:45163)
0806 22:37:28.338961 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.338970 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:28.339022 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:28.339214 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:28.339215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:28.339218 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:28.339818 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:28.339822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.340530 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.340531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:28.341549 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:28.341551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.342291 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.342293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:28.342369 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.342373 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:28.342456 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:37:28.342493 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":18,"thread_start_us":220,"threads_started":1}
I20260806 22:37:28.342588 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.338815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:45163 (local address 127.16.235.66:35115)
0806 22:37:28.338993 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:28.338996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:28.339013 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:37:28.339080 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:28.339080 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:28.339094 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:28.339174 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:28.339915 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.340436 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.341626 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.342194 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.342266 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.342275 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.342465 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.342513 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260806 22:37:28.342635 19981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ac3a2ee00342a2ad1c0a57afff1dac" candidate_uuid: "5356bf3eed2e4605b595e3299dc1adb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58170824e82419d9c5b35917150641b" is_pre_election: true
I20260806 22:37:28.342775 19981 raft_consensus.cc:2454] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5356bf3eed2e4605b595e3299dc1adb5 in term 0.
I20260806 22:37:28.342878 20171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ac3a2ee00342a2ad1c0a57afff1dac" candidate_uuid: "5356bf3eed2e4605b595e3299dc1adb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c22fa0120d4004a0cd70dbb7904078" is_pre_election: true
I20260806 22:37:28.343001 20171 raft_consensus.cc:2454] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5356bf3eed2e4605b595e3299dc1adb5 in term 0.
I20260806 22:37:28.342979 20280 leader_election.cc:302] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5356bf3eed2e4605b595e3299dc1adb5, e58170824e82419d9c5b35917150641b; no voters:
I20260806 22:37:28.343114 20599 raft_consensus.cc:2781] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:28.343158 20599 raft_consensus.cc:482] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:28.343183 20599 raft_consensus.cc:3037] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:28.343973 20599 raft_consensus.cc:504] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.344141 20599 leader_election.cc:288] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [CANDIDATE]: Term 1 election: Requested vote from peers e58170824e82419d9c5b35917150641b (127.16.235.65:36347), d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66:35115)
I20260806 22:37:28.344295 19981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ac3a2ee00342a2ad1c0a57afff1dac" candidate_uuid: "5356bf3eed2e4605b595e3299dc1adb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58170824e82419d9c5b35917150641b"
I20260806 22:37:28.344322 20171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ac3a2ee00342a2ad1c0a57afff1dac" candidate_uuid: "5356bf3eed2e4605b595e3299dc1adb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c22fa0120d4004a0cd70dbb7904078"
I20260806 22:37:28.344379 19981 raft_consensus.cc:3037] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:28.344396 20171 raft_consensus.cc:3037] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:28.345186 19981 raft_consensus.cc:2454] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5356bf3eed2e4605b595e3299dc1adb5 in term 1.
I20260806 22:37:28.345186 20171 raft_consensus.cc:2454] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5356bf3eed2e4605b595e3299dc1adb5 in term 1.
I20260806 22:37:28.345403 20280 leader_election.cc:302] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5356bf3eed2e4605b595e3299dc1adb5, e58170824e82419d9c5b35917150641b; no voters:
I20260806 22:37:28.345533 20599 raft_consensus.cc:2781] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:28.345723 20599 raft_consensus.cc:686] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 LEADER]: Becoming Leader. State: Replica: 5356bf3eed2e4605b595e3299dc1adb5, State: Running, Role: LEADER
I20260806 22:37:28.345805 20599 consensus_queue.cc:237] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.346578 19756 catalog_manager.cc:5495] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67). New cstate: current_term: 1 leader_uuid: "5356bf3eed2e4605b595e3299dc1adb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } health_report { overall_health: HEALTHY } } }
I20260806 22:37:28.383710 20613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.380930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.16.235.67:45809)
0806 22:37:28.381136 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:28.381138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:28.381146 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:28.381197 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:28.381198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:28.381225 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:28.381320 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:28.381454 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.381966 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.382809 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.383062 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.383140 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.383303 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:28.383431 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:28.383530 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.383573 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.383595 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":115,"threads_started":1}
I20260806 22:37:28.389302 20617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.386751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35115 (local address 127.16.235.67:55347)
0806 22:37:28.386931 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.386942 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:28.387012 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:28.387245 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:28.387246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:28.387248 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:28.387302 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:28.387304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.387931 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.387932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:28.388522 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:28.388524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.388928 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.388929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:28.388983 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.388998 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:28.389162 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:28.389164 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:37:28.389195 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 22:37:28.389233 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260806 22:37:28.389295 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.387023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:55347 (local address 127.16.235.66:35115)
0806 22:37:28.387112 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:28.387114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:28.387123 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:28.387149 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:28.387150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:28.387160 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:28.387229 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:28.387367 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.387862 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.388632 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.388851 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.388917 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.389076 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:28.389106 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:28.389147 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.389215 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.389238 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:37:28.392235 20171 raft_consensus.cc:1261] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Refusing update from remote peer 5356bf3eed2e4605b595e3299dc1adb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:28.392233 19981 raft_consensus.cc:1261] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Refusing update from remote peer 5356bf3eed2e4605b595e3299dc1adb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:28.392544 20605 consensus_queue.cc:1035] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:28.392697 20599 consensus_queue.cc:1035] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:28.401192 20613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.398766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:36969 (local address 127.16.235.67:45809)
0806 22:37:28.398815 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:28.398817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:28.398867 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:37:28.398938 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:28.398939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:28.398950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:28.399002 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:28.399134 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.399626 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.400468 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.400770 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.400856 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.401002 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:28.401030 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:28.401071 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.401106 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.401125 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:37:28.401204 20623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.398545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:45809 (local address 127.16.235.66:36969)
0806 22:37:28.398812 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:28.398825 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:28.398886 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:28.399004 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:28.399005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:28.399007 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:28.399064 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:28.399066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.399738 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.399739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:28.400348 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:28.400351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.400855 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.400856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:28.400927 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.400933 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:28.401073 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:28.401074 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:37:28.401105 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 22:37:28.401144 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":203,"threads_started":1}
I20260806 22:37:28.407619 19756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:28.407804 19756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:28.408480 19756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a: Sending DeleteTablet for 3 replicas of tablet 84ac3a2ee00342a2ad1c0a57afff1dac
I20260806 22:37:28.408814 19953 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ac3a2ee00342a2ad1c0a57afff1dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:28 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35338
I20260806 22:37:28.408941 20572 tablet_replica.cc:330] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: stopping tablet replica
I20260806 22:37:28.409010 20572 raft_consensus.cc:2229] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:28.409061 20572 raft_consensus.cc:2258] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:28.409229 20341 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ac3a2ee00342a2ad1c0a57afff1dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:28 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35814
I20260806 22:37:28.409328 20576 tablet_replica.cc:330] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: stopping tablet replica
I20260806 22:37:28.409379 20576 raft_consensus.cc:2229] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:28.409372 20572 ts_tablet_manager.cc:1905] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:28.409476 20576 raft_consensus.cc:2258] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:28.409847 20576 ts_tablet_manager.cc:1905] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:28.411337 20572 ts_tablet_manager.cc:1918] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:28.411417 20572 log.cc:1196] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/84ac3a2ee00342a2ad1c0a57afff1dac
I20260806 22:37:28.411695 20572 ts_tablet_manager.cc:1939] T 84ac3a2ee00342a2ad1c0a57afff1dac P e58170824e82419d9c5b35917150641b: Deleting consensus metadata
I20260806 22:37:28.411617 20576 ts_tablet_manager.cc:1918] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:28.411767 20576 log.cc:1196] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/84ac3a2ee00342a2ad1c0a57afff1dac
I20260806 22:37:28.412030 20576 ts_tablet_manager.cc:1939] T 84ac3a2ee00342a2ad1c0a57afff1dac P 5356bf3eed2e4605b595e3299dc1adb5: Deleting consensus metadata
I20260806 22:37:28.412348 19737 catalog_manager.cc:4841] TS e58170824e82419d9c5b35917150641b (127.16.235.65:36347): tablet 84ac3a2ee00342a2ad1c0a57afff1dac (table test-table [id=f8b5061a17ab4e56ba8c965291cdb7c4]) successfully deleted
I20260806 22:37:28.412369 19735 catalog_manager.cc:4841] TS 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809): tablet 84ac3a2ee00342a2ad1c0a57afff1dac (table test-table [id=f8b5061a17ab4e56ba8c965291cdb7c4]) successfully deleted
I20260806 22:37:28.412477 20150 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ac3a2ee00342a2ad1c0a57afff1dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:28 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47548
I20260806 22:37:28.412590 20574 tablet_replica.cc:330] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: stopping tablet replica
I20260806 22:37:28.412655 20574 raft_consensus.cc:2229] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:28.412698 20574 raft_consensus.cc:2258] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:28.413062 20574 ts_tablet_manager.cc:1905] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:28.412997 19756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:28.413172 19756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:28.414577 19953 tablet_service.cc:1467] Processing CreateTablet for tablet caf669fdaa63452492822fd3f0e399ed (DEFAULT_TABLE table=test-table [id=398bf34ba5d64cc4a74e7881a02555df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:28.414749 19953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf669fdaa63452492822fd3f0e399ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:28.414821 20574 ts_tablet_manager.cc:1918] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:28.414885 20574 log.cc:1196] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/84ac3a2ee00342a2ad1c0a57afff1dac
I20260806 22:37:28.415115 20574 ts_tablet_manager.cc:1939] T 84ac3a2ee00342a2ad1c0a57afff1dac P d7c22fa0120d4004a0cd70dbb7904078: Deleting consensus metadata
I20260806 22:37:28.415486 19735 catalog_manager.cc:4841] TS d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66:35115): tablet 84ac3a2ee00342a2ad1c0a57afff1dac (table test-table [id=f8b5061a17ab4e56ba8c965291cdb7c4]) successfully deleted
I20260806 22:37:28.415951 20592 tablet_bootstrap.cc:492] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b: Bootstrap starting.
I20260806 22:37:28.416074 20341 tablet_service.cc:1467] Processing CreateTablet for tablet caf669fdaa63452492822fd3f0e399ed (DEFAULT_TABLE table=test-table [id=398bf34ba5d64cc4a74e7881a02555df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:28.416209 20341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf669fdaa63452492822fd3f0e399ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:28.416324 20592 tablet_bootstrap.cc:654] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:28.416986 20592 tablet_bootstrap.cc:492] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b: No bootstrap required, opened a new log
I20260806 22:37:28.417050 20592 ts_tablet_manager.cc:1397] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:28.417179 20592 raft_consensus.cc:348] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.417259 20592 raft_consensus.cc:374] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:28.417280 20592 raft_consensus.cc:729] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58170824e82419d9c5b35917150641b, State: Initialized, Role: FOLLOWER
I20260806 22:37:28.417340 20592 consensus_queue.cc:260] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.417451 20592 ts_tablet_manager.cc:1428] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:28.417536 20591 tablet_bootstrap.cc:492] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: Bootstrap starting.
I20260806 22:37:28.417920 20591 tablet_bootstrap.cc:654] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:28.418536 20150 tablet_service.cc:1467] Processing CreateTablet for tablet caf669fdaa63452492822fd3f0e399ed (DEFAULT_TABLE table=test-table [id=398bf34ba5d64cc4a74e7881a02555df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:28.418699 20150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf669fdaa63452492822fd3f0e399ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:28.419714 20590 tablet_bootstrap.cc:492] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078: Bootstrap starting.
I20260806 22:37:28.419988 20591 tablet_bootstrap.cc:492] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: No bootstrap required, opened a new log
I20260806 22:37:28.420025 20591 ts_tablet_manager.cc:1397] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:37:28.420096 20590 tablet_bootstrap.cc:654] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:28.420156 20591 raft_consensus.cc:348] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.420208 20591 raft_consensus.cc:374] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:28.420221 20591 raft_consensus.cc:729] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5356bf3eed2e4605b595e3299dc1adb5, State: Initialized, Role: FOLLOWER
I20260806 22:37:28.420302 20591 consensus_queue.cc:260] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.420436 20591 ts_tablet_manager.cc:1428] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:28.421216 20590 tablet_bootstrap.cc:492] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078: No bootstrap required, opened a new log
I20260806 22:37:28.421271 20590 ts_tablet_manager.cc:1397] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:28.421412 20590 raft_consensus.cc:348] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.421478 20590 raft_consensus.cc:374] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:28.421499 20590 raft_consensus.cc:729] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c22fa0120d4004a0cd70dbb7904078, State: Initialized, Role: FOLLOWER
I20260806 22:37:28.421550 20590 consensus_queue.cc:260] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.421638 20590 ts_tablet_manager.cc:1428] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:28.470906 20549 raft_consensus.cc:482] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:28.471019 20549 raft_consensus.cc:504] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.471258 20549 leader_election.cc:288] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e58170824e82419d9c5b35917150641b (127.16.235.65:36347), 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809)
I20260806 22:37:28.471478 19981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf669fdaa63452492822fd3f0e399ed" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58170824e82419d9c5b35917150641b" is_pre_election: true
I20260806 22:37:28.471524 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf669fdaa63452492822fd3f0e399ed" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5356bf3eed2e4605b595e3299dc1adb5" is_pre_election: true
I20260806 22:37:28.471604 19981 raft_consensus.cc:2454] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 0.
I20260806 22:37:28.471633 20365 raft_consensus.cc:2454] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 0.
I20260806 22:37:28.471794 20083 leader_election.cc:302] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7c22fa0120d4004a0cd70dbb7904078, e58170824e82419d9c5b35917150641b; no voters:
I20260806 22:37:28.471912 20549 raft_consensus.cc:2781] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:28.471951 20549 raft_consensus.cc:482] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:28.471983 20549 raft_consensus.cc:3037] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:28.472812 20549 raft_consensus.cc:504] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.472975 20549 leader_election.cc:288] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 election: Requested vote from peers e58170824e82419d9c5b35917150641b (127.16.235.65:36347), 5356bf3eed2e4605b595e3299dc1adb5 (127.16.235.67:45809)
I20260806 22:37:28.473148 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf669fdaa63452492822fd3f0e399ed" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5356bf3eed2e4605b595e3299dc1adb5"
I20260806 22:37:28.473141 19981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf669fdaa63452492822fd3f0e399ed" candidate_uuid: "d7c22fa0120d4004a0cd70dbb7904078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58170824e82419d9c5b35917150641b"
I20260806 22:37:28.473270 19981 raft_consensus.cc:3037] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:28.473271 20365 raft_consensus.cc:3037] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:28.473928 20365 raft_consensus.cc:2454] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 1.
I20260806 22:37:28.474123 19981 raft_consensus.cc:2454] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7c22fa0120d4004a0cd70dbb7904078 in term 1.
I20260806 22:37:28.474118 20078 leader_election.cc:302] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5356bf3eed2e4605b595e3299dc1adb5, d7c22fa0120d4004a0cd70dbb7904078; no voters:
I20260806 22:37:28.474241 20549 raft_consensus.cc:2781] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:28.474288 20549 raft_consensus.cc:686] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 1 LEADER]: Becoming Leader. State: Replica: d7c22fa0120d4004a0cd70dbb7904078, State: Running, Role: LEADER
I20260806 22:37:28.474365 20549 consensus_queue.cc:237] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } }
I20260806 22:37:28.475122 19756 catalog_manager.cc:5495] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7c22fa0120d4004a0cd70dbb7904078 (127.16.235.66). New cstate: current_term: 1 leader_uuid: "d7c22fa0120d4004a0cd70dbb7904078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7c22fa0120d4004a0cd70dbb7904078" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:28.487257 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:28.484652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56804 (local address 127.16.235.66:35115)
0806 22:37:28.484713 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:28.484715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:28.484723 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:28.484818 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:28.484818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:28.484829 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:28.484875 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:28.485000 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.485482 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.486541 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:28.486793 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:28.486851 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:28.487019 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:28.487047 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:28.487086 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:28.487143 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:37:28.487163 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:37:28.489740 19981 raft_consensus.cc:1261] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Refusing update from remote peer d7c22fa0120d4004a0cd70dbb7904078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:28.489923 20365 raft_consensus.cc:1261] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Refusing update from remote peer d7c22fa0120d4004a0cd70dbb7904078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:28.490034 20549 consensus_queue.cc:1035] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e58170824e82419d9c5b35917150641b" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 36347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:28.490139 20545 consensus_queue.cc:1035] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5356bf3eed2e4605b595e3299dc1adb5" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:37:28.495913 20547 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:37:30.017854 19755 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:30.017969 19755 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:30.018895 19755 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5c990f0635d420a96f3a83f4b93255a: Sending DeleteTablet for 3 replicas of tablet caf669fdaa63452492822fd3f0e399ed
I20260806 22:37:30.019187 19953 tablet_service.cc:1514] Processing DeleteTablet for tablet caf669fdaa63452492822fd3f0e399ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:30 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35338
I20260806 22:37:30.019210 20341 tablet_service.cc:1514] Processing DeleteTablet for tablet caf669fdaa63452492822fd3f0e399ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:30 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35814
I20260806 22:37:30.019217 20147 tablet_service.cc:1514] Processing DeleteTablet for tablet caf669fdaa63452492822fd3f0e399ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:30 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47548
I20260806 22:37:30.019410 20661 tablet_replica.cc:330] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b: stopping tablet replica
I20260806 22:37:30.019474 20661 raft_consensus.cc:2229] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:30.019536 20661 raft_consensus.cc:2258] T caf669fdaa63452492822fd3f0e399ed P e58170824e82419d9c5b35917150641b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:30.019613 20663 tablet_replica.cc:330] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078: stopping tablet replica
I20260806 22:37:30.019726 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 19860
I20260806 22:37:30.027675 20663 raft_consensus.cc:2229] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:30.027837 20663 raft_consensus.cc:2258] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:30.028169 20663 ts_tablet_manager.cc:1905] T caf669fdaa63452492822fd3f0e399ed P d7c22fa0120d4004a0cd70dbb7904078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:30.029239 20662 tablet_replica.cc:330] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: stopping tablet replica
I20260806 22:37:30.029320 20662 raft_consensus.cc:2229] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:30.029366 20662 raft_consensus.cc:2258] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:30.029670 20662 ts_tablet_manager.cc:1905] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 22:37:30.039085 19737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:36347 (error 108)
I20260806 22:37:30.039266 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 20055
I20260806 22:37:30.047109 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:30.039672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:36347 (local address 127.0.0.1:41254)
0806 22:37:30.046955 (+ 7283us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:30.047055 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7214,"thread_start_us":63,"threads_started":1}
W20260806 22:37:30.047281 19737 catalog_manager.cc:4584] TS e58170824e82419d9c5b35917150641b (127.16.235.65:36347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet caf669fdaa63452492822fd3f0e399ed: Network error: Client connection negotiation failed: client connection to 127.16.235.65:36347: connect: Connection refused (error 111)
I20260806 22:37:30.089135 20662 ts_tablet_manager.cc:1918] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:37:30.089255 20662 log.cc:1196] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/caf669fdaa63452492822fd3f0e399ed
I20260806 22:37:30.089547 20662 ts_tablet_manager.cc:1939] T caf669fdaa63452492822fd3f0e399ed P 5356bf3eed2e4605b595e3299dc1adb5: Deleting consensus metadata
I20260806 22:37:30.094215 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 20257
I20260806 22:37:30.096693 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:30.094146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35115 (local address 127.0.0.1:56820)
0806 22:37:30.094276 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:30.094357 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.66:35115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 22:37:30.104735 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:30.104569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:36347 (local address 127.0.0.1:41260)
0806 22:37:30.104641 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:30.104677 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:37:30.105281 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:30.105141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:36347 (local address 127.0.0.1:41262)
0806 22:37:30.105207 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:30.105241 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:36347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:37:30.112128 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 19712
I20260806 22:37:30.112177 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:30.112030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:45809 (local address 127.0.0.1:59882)
0806 22:37:30.112099 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:30.112131 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.67:45809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
2026-08-06T22:37:30Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6273 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:30 dist-test-slave-3kk6 krb5kdc[20669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:30 dist-test-slave-3kk6 krb5kdc[20669](info): set up 2 sockets
Aug 06 22:37:30 dist-test-slave-3kk6 krb5kdc[20669](info): commencing operation
krb5kdc: starting...
W20260806 22:37:31.795171 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.654s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:31 dist-test-slave-3kk6 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:31.925905 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45949
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:45949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:32.012131 20686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:32.012374 20686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:32.012401 20686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:32.013549 20686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:32.013589 20686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:32.013604 20686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:32.013617 20686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:32.013629 20686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:32.015172 20686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:45949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:32.015403 20686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:32.015610 20686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:32.021091 20693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.021087 20695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.021087 20697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:32.021183 20686 server_base.cc:1034] running on GCE node
I20260806 22:37:32.021729 20686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:32.022094 20686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:32.023273 20686 hybrid_clock.cc:648] HybridClock initialized: now 1786055852023252 us; error 56 us; skew 500 ppm
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:32.026049 20686 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:32.026554 20686 webserver.cc:460] Webserver started at http://127.16.235.126:46277/ using document root <none> and password file <none>
I20260806 22:37:32.026857 20686 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:32.026935 20686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:32.027100 20686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:32.028271 20686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "6c9ec3c745f5ffea5b0f276e7dafafd4"
server_key_iv: "b8bbfedbb047e4cb9c40a7cb3b8ffddb"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.028674 20686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "6c9ec3c745f5ffea5b0f276e7dafafd4"
server_key_iv: "b8bbfedbb047e4cb9c40a7cb3b8ffddb"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.030158 20686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:32.031029 20705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.031211 20686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:32.031323 20686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "6c9ec3c745f5ffea5b0f276e7dafafd4"
server_key_iv: "b8bbfedbb047e4cb9c40a7cb3b8ffddb"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.031418 20686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:32.049137 20686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:32.051478 20686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:32.051685 20686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:32.056242 20686 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:45949
I20260806 22:37:32.056296 20775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:45949 every 8 connection(s)
I20260806 22:37:32.056946 20686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:32.057783 20776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:32.060632 20776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3: Bootstrap starting.
I20260806 22:37:32.061236 20776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:32.061489 20776 log.cc:824] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:32.062091 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 20686
I20260806 22:37:32.062170 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:32.062304 20776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3: No bootstrap required, opened a new log
I20260806 22:37:32.062407 17325 external_mini_cluster.cc:1428] Setting key 46b4e9ed6fdfd5c071250d44578585fe
I20260806 22:37:32.064802 20776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45949 } }
I20260806 22:37:32.065024 20776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:32.065068 20776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c00ca4a00fb40c9ac22a7ee34a746c3, State: Initialized, Role: FOLLOWER
I20260806 22:37:32.065236 20776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45949 } }
I20260806 22:37:32.065346 20776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:32.065394 20776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:32.065450 20776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:32.066278 20776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45949 } }
I20260806 22:37:32.066397 20776 leader_election.cc:302] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c00ca4a00fb40c9ac22a7ee34a746c3; no voters:
I20260806 22:37:32.066565 20776 leader_election.cc:288] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:32.066639 20786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:32.066833 20786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [term 1 LEADER]: Becoming Leader. State: Replica: 0c00ca4a00fb40c9ac22a7ee34a746c3, State: Running, Role: LEADER
I20260806 22:37:32.066856 20776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:32.066928 20786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45949 } }
I20260806 22:37:32.067284 20787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45949 } } }
I20260806 22:37:32.067274 20786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c00ca4a00fb40c9ac22a7ee34a746c3. Latest consensus state: current_term: 1 leader_uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00ca4a00fb40c9ac22a7ee34a746c3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45949 } } }
I20260806 22:37:32.067384 20787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:32.067391 20786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:32.067662 20797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:32.068189 20797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:32.070060 20797 catalog_manager.cc:1347] Generated new cluster ID: 9820cf27769645fd81b2682b10e96137
I20260806 22:37:32.070123 20797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:32.075744 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.063030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.16.235.126:45949)
0806 22:37:32.063281 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:32.063292 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:32.063326 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:37:32.063972 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:32.063983 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:32.064054 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:32.064232 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:32.064896 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.066831 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.068126 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.068792 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.068910 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.071759 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:32.071763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:32.071763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:32.071777 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:32.073636 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.074044 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.074048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.074048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.074145 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.074379 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.074380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.074381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.074636 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:32.075389 (+ 753us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:32.075466 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:37:32.075583 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":114,"threads_started":1}
I20260806 22:37:32.079411 20797 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:32.080085 20797 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:32.085525 20797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c00ca4a00fb40c9ac22a7ee34a746c3: Generated new TSK 0
I20260806 22:37:32.085760 20797 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:32.121492 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45949
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:32.208985 20815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:32.209270 20815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:32.209314 20815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:32.210561 20815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:32.210640 20815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:32.210711 20815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:32.212309 20815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:32.212613 20815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:32.212903 20815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:32.218398 20826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.218413 20822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:32.218621 20815 server_base.cc:1034] running on GCE node
W20260806 22:37:32.218430 20821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:32.218961 20815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:32.219269 20815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:32.220458 20815 hybrid_clock.cc:648] HybridClock initialized: now 1786055852220444 us; error 29 us; skew 500 ppm
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:32.223161 20815 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:32.223625 20815 webserver.cc:460] Webserver started at http://127.16.235.65:42589/ using document root <none> and password file <none>
I20260806 22:37:32.223904 20815 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:32.223976 20815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:32.224135 20815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:32.225270 20815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "865b6b7fe5aa48d4ae27848026f7819c"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "ef322d74416a9784ef362676c0857c82"
server_key_iv: "20ffdc54546737e59fe3893f6129e0bd"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.225661 20815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "865b6b7fe5aa48d4ae27848026f7819c"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "ef322d74416a9784ef362676c0857c82"
server_key_iv: "20ffdc54546737e59fe3893f6129e0bd"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.227183 20815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:32.228024 20835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.228211 20815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:32.228324 20815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "865b6b7fe5aa48d4ae27848026f7819c"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "ef322d74416a9784ef362676c0857c82"
server_key_iv: "20ffdc54546737e59fe3893f6129e0bd"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.228405 20815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:32.269582 20815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:32.271821 20815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:32.272008 20815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:32.272284 20815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:32.272622 20815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:32.272677 20815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.272720 20815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:32.272763 20815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.279079 20815 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:45205
I20260806 22:37:32.279137 20990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:45205 every 8 connection(s)
I20260806 22:37:32.279491 20815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:32.288591 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 20815
I20260806 22:37:32.288724 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:32.288955 17325 external_mini_cluster.cc:1428] Setting key c518075e6b40bdaec51c0c5ceaaf56a8
I20260806 22:37:32.289496 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.280242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:51277 (local address 127.16.235.126:45949)
0806 22:37:32.280301 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:32.280303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:32.280537 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:37:32.281048 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:32.281050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:32.281063 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:32.281140 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:32.282139 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.282815 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.283790 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.284009 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.284076 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.286375 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:32.286376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:32.286377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:32.286387 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:32.287840 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.288331 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.288333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.288333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.288414 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.288704 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.288706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.288706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.288875 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:32.288962 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:32.289316 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 22:37:32.289424 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:37:32.289701 20995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.280208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45949 (local address 127.16.235.65:51277)
0806 22:37:32.280380 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.280426 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:32.280939 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:32.281193 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:32.281196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:32.281783 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:32.282014 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.282019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.282905 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.282906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.283699 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.283701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.284095 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.284095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.284195 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.284704 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:32.284712 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:32.286259 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:32.287940 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.287949 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.287952 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.288225 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.288498 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.288499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.288499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.288617 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.288952 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:32.288957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:32.289124 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:37:32.289262 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 22:37:32.289469 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
I20260806 22:37:32.290195 20992 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45949
I20260806 22:37:32.290298 20992 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:32.290510 20992 heartbeater.cc:511] Master 127.16.235.126:45949 requested a full tablet report, sending...
I20260806 22:37:32.291088 20723 ts_manager.cc:194] Registered new tserver with Master: 865b6b7fe5aa48d4ae27848026f7819c (127.16.235.65:45205)
I20260806 22:37:32.291729 20723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:51277
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:32.315802 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45949
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:32.401794 21000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:32.402027 21000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:32.402052 21000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:32.403180 21000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:32.403218 21000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:32.403245 21000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:32.404630 21000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:32.404830 21000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:32.405031 21000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:32.410231 21008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.410240 21005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:32.410460 21000 server_base.cc:1034] running on GCE node
W20260806 22:37:32.410271 21006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:32.410804 21000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:32.411088 21000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:32.412261 21000 hybrid_clock.cc:648] HybridClock initialized: now 1786055852412233 us; error 46 us; skew 500 ppm
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:32.414994 21000 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:32.415513 21000 webserver.cc:460] Webserver started at http://127.16.235.66:38305/ using document root <none> and password file <none>
I20260806 22:37:32.415752 21000 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:32.415798 21000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:32.415961 21000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:32.417105 21000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "62d36d7f54a84207bbd8537524e84fb8"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "3c864f5deb8adb0c7166d617bf23496b"
server_key_iv: "d089dfc717b0cbe1c55a21be6b0fbce5"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.417584 21000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "62d36d7f54a84207bbd8537524e84fb8"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "3c864f5deb8adb0c7166d617bf23496b"
server_key_iv: "d089dfc717b0cbe1c55a21be6b0fbce5"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.419070 21000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:32.419842 21016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.420053 21000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:32.420181 21000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "62d36d7f54a84207bbd8537524e84fb8"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "3c864f5deb8adb0c7166d617bf23496b"
server_key_iv: "d089dfc717b0cbe1c55a21be6b0fbce5"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.420266 21000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:32.434139 21000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:32.436457 21000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:32.436643 21000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:32.436918 21000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:32.437274 21000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:32.437331 21000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.437376 21000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:32.437402 21000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.443964 21000 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:39859
I20260806 22:37:32.444024 21179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:39859 every 8 connection(s)
I20260806 22:37:32.444410 21000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:32.451974 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 21000
I20260806 22:37:32.452100 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:32.452293 17325 external_mini_cluster.cc:1428] Setting key 16ac6577c1a0f1265b4cfc3d95096341
I20260806 22:37:32.455111 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.445342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:57715 (local address 127.16.235.126:45949)
0806 22:37:32.445420 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:32.445423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:32.445641 (+ 218us) server_negotiation.cc:407] Connection header received
0806 22:37:32.446194 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:32.446196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:32.446207 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:32.446281 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:32.447566 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.448124 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.449176 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.449378 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.449439 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.451906 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:32.451907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:32.451908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:32.451915 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:32.453445 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.453965 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.453966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.453966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.454021 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.454310 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.454310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.454311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.454416 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:32.454467 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:32.454874 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 22:37:32.454975 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:37:32.455292 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.445314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45949 (local address 127.16.235.66:57715)
0806 22:37:32.445494 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.445542 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:32.446069 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:32.446457 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:32.446461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:32.447216 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:32.447433 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.447438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.448227 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.448228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.449087 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.449089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.449467 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.449468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.449575 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.450208 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:32.450217 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:32.451788 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:32.453531 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.453539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.453543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.453886 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.454090 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.454091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.454092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.454245 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.454483 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:32.454490 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:32.454702 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 22:37:32.454831 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 22:37:32.455049 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260806 22:37:32.455735 21182 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45949
I20260806 22:37:32.455885 21182 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:32.456103 21182 heartbeater.cc:511] Master 127.16.235.126:45949 requested a full tablet report, sending...
I20260806 22:37:32.456511 20723 ts_manager.cc:194] Registered new tserver with Master: 62d36d7f54a84207bbd8537524e84fb8 (127.16.235.66:39859)
I20260806 22:37:32.456897 20723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:57715
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:32.481411 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45949
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:32.568334 21187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:32.568578 21187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:32.568604 21187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:32.569854 21187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:32.569905 21187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:32.569934 21187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:32.571487 21187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:32.571777 21187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:32.572050 21187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:32.577365 21199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.577365 21197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:32.577395 21196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:32.577564 21187 server_base.cc:1034] running on GCE node
I20260806 22:37:32.577780 21187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:32.577998 21187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:32.579205 21187 hybrid_clock.cc:648] HybridClock initialized: now 1786055852579194 us; error 40 us; skew 500 ppm
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:32.582098 21187 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:32.582636 21187 webserver.cc:460] Webserver started at http://127.16.235.67:41087/ using document root <none> and password file <none>
I20260806 22:37:32.582906 21187 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:32.582954 21187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:32.583056 21187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:32.584123 21187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "bc8500be02b74532815288283dc0f367"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "d4a3a12800c2b703e48635c2997b29b2"
server_key_iv: "2a44f1b05aab29497770e24752bf1c83"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.584474 21187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "bc8500be02b74532815288283dc0f367"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "d4a3a12800c2b703e48635c2997b29b2"
server_key_iv: "2a44f1b05aab29497770e24752bf1c83"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.585878 21187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:32.586807 21207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.587037 21187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:32.587149 21187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "bc8500be02b74532815288283dc0f367"
format_stamp: "Formatted at 2026-08-06 22:37:32 on dist-test-slave-3kk6"
server_key: "d4a3a12800c2b703e48635c2997b29b2"
server_key_iv: "2a44f1b05aab29497770e24752bf1c83"
server_key_version: "encryptionkey@0"
I20260806 22:37:32.587236 21187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:32.609951 21187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:32.612262 21187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:32.612437 21187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:32.612712 21187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:32.613057 21187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:32.613092 21187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.613116 21187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:32.613161 21187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:32.619552 21187 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:42885
I20260806 22:37:32.619647 21360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:42885 every 8 connection(s)
I20260806 22:37:32.619961 21187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:32.627910 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 21187
I20260806 22:37:32.628026 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:32.628229 17325 external_mini_cluster.cc:1428] Setting key fe898b022ae89d29ceac1fe8b3510398
I20260806 22:37:32.629880 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.620798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:34027 (local address 127.16.235.126:45949)
0806 22:37:32.620865 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:32.620867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:32.621069 (+ 202us) server_negotiation.cc:407] Connection header received
0806 22:37:32.621561 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:32.621562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:32.621573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:32.621630 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:32.622571 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.623205 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.624261 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.624525 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.624586 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.626911 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:32.626912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:32.626912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:32.626920 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:32.628389 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.628920 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.628921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.628922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.628986 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:32.629297 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:32.629298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:32.629299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:32.629435 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:32.629491 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:32.629743 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 22:37:32.629812 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:37:32.630084 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:32.620771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45949 (local address 127.16.235.67:34027)
0806 22:37:32.620937 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:32.620981 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:32.621476 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:32.621693 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:32.621696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:32.622280 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:32.622479 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.622484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.623359 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.623360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.624185 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:32.624187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:32.624630 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:32.624632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:32.624749 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:32.625361 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:32.625370 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:32.626822 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:32.628475 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.628483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.628486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.628814 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.629065 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:32.629065 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:32.629066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:32.629215 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:32.629498 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:32.629504 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:32.629626 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 22:37:32.629716 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:37:32.629858 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260806 22:37:32.630559 21362 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45949
I20260806 22:37:32.630692 21362 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:32.630905 21362 heartbeater.cc:511] Master 127.16.235.126:45949 requested a full tablet report, sending...
I20260806 22:37:32.631294 20723 ts_manager.cc:194] Registered new tserver with Master: bc8500be02b74532815288283dc0f367 (127.16.235.67:42885)
I20260806 22:37:32.631690 20723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:34027
I20260806 22:37:32.640211 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:37:32.653841 20785 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:32.652801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.16.235.126:45949)
0806 22:37:32.652876 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:32.652878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:32.652917 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:37:32.652999 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:32.653001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:32.653010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:32.653075 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:32.653698 (+ 623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46064: BlockingRecv error: recv got EOF from 127.0.0.1:46064 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:37:32.654737 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 20815
I20260806 22:37:32.665952 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 21000
I20260806 22:37:32.676163 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 21187
I20260806 22:37:32.703333 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 20686
2026-08-06T22:37:32Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2588 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[21378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[21378](info): set up 2 sockets
Aug 06 22:37:32 dist-test-slave-3kk6 krb5kdc[21378](info): commencing operation
krb5kdc: starting...
W20260806 22:37:33.934866 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:34 dist-test-slave-3kk6 krb5kdc[21378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:34.065879 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38701
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:37961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:38701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:34.156836 21395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:34.157094 21395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:34.157119 21395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:34.158262 21395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:34.158300 21395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:34.158314 21395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:34.158327 21395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:34.158340 21395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:34.159909 21395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:37961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:38701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:34.160233 21395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:34.160531 21395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:34.166106 21407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:34.166090 21403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:34.166126 21395 server_base.cc:1034] running on GCE node
W20260806 22:37:34.166091 21402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:34.166565 21395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:34.166956 21395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:34.168145 21395 hybrid_clock.cc:648] HybridClock initialized: now 1786055854168138 us; error 44 us; skew 500 ppm
Aug 06 22:37:34 dist-test-slave-3kk6 krb5kdc[21378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055854, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:34.171128 21395 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:34.171741 21395 webserver.cc:460] Webserver started at http://127.16.235.126:42373/ using document root <none> and password file <none>
I20260806 22:37:34.172021 21395 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:34.172098 21395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:34.172266 21395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:34.173452 21395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "7ecfb4a489eb42e5a37631abad9429af"
format_stamp: "Formatted at 2026-08-06 22:37:34 on dist-test-slave-3kk6"
server_key: "9f4a764a22ce29748e3dd44bb93d6581"
server_key_iv: "76b2222af9a63fff94edc4fc8cb8cfc4"
server_key_version: "encryptionkey@0"
I20260806 22:37:34.173866 21395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "7ecfb4a489eb42e5a37631abad9429af"
format_stamp: "Formatted at 2026-08-06 22:37:34 on dist-test-slave-3kk6"
server_key: "9f4a764a22ce29748e3dd44bb93d6581"
server_key_iv: "76b2222af9a63fff94edc4fc8cb8cfc4"
server_key_version: "encryptionkey@0"
I20260806 22:37:34.175588 21395 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:37:34.176431 21416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:34.176630 21395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:34.176748 21395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "7ecfb4a489eb42e5a37631abad9429af"
format_stamp: "Formatted at 2026-08-06 22:37:34 on dist-test-slave-3kk6"
server_key: "9f4a764a22ce29748e3dd44bb93d6581"
server_key_iv: "76b2222af9a63fff94edc4fc8cb8cfc4"
server_key_version: "encryptionkey@0"
I20260806 22:37:34.176846 21395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:34.217145 21395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:34.219580 21395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:34.219785 21395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:34.224599 21395 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:38701
I20260806 22:37:34.224639 21486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:38701 every 8 connection(s)
I20260806 22:37:34.225193 21395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:34.226043 21488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:34.228600 21488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af: Bootstrap starting.
I20260806 22:37:34.229310 21488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:34.229660 21488 log.cc:824] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:34.230444 21488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af: No bootstrap required, opened a new log
I20260806 22:37:34.232235 21488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecfb4a489eb42e5a37631abad9429af" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38701 } }
I20260806 22:37:34.232396 21488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:34.232448 21488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ecfb4a489eb42e5a37631abad9429af, State: Initialized, Role: FOLLOWER
I20260806 22:37:34.232621 21488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecfb4a489eb42e5a37631abad9429af" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38701 } }
I20260806 22:37:34.232717 21488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:34.232758 21488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:34.232846 21488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:34.232838 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 21395
I20260806 22:37:34.232926 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:34.233105 17325 external_mini_cluster.cc:1428] Setting key b5605c6008e4035ea417fe6193174fab
I20260806 22:37:34.234277 21488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecfb4a489eb42e5a37631abad9429af" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38701 } }
I20260806 22:37:34.234402 21488 leader_election.cc:302] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ecfb4a489eb42e5a37631abad9429af; no voters:
I20260806 22:37:34.234575 21488 leader_election.cc:288] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:34.234638 21497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:34.234822 21497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [term 1 LEADER]: Becoming Leader. State: Replica: 7ecfb4a489eb42e5a37631abad9429af, State: Running, Role: LEADER
I20260806 22:37:34.234925 21488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:34.234922 21497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecfb4a489eb42e5a37631abad9429af" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38701 } }
I20260806 22:37:34.235280 21497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ecfb4a489eb42e5a37631abad9429af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecfb4a489eb42e5a37631abad9429af" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38701 } } }
I20260806 22:37:34.235378 21497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:34.235520 21497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ecfb4a489eb42e5a37631abad9429af. Latest consensus state: current_term: 1 leader_uuid: "7ecfb4a489eb42e5a37631abad9429af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ecfb4a489eb42e5a37631abad9429af" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38701 } } }
I20260806 22:37:34.235571 21497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:34.235737 21507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:34.236413 21507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:34.238099 21523 mvcc.cc:205] Tried to move back new op lower bound from 7315684778957299712 to 7315684778946740224. Current Snapshot: MvccSnapshot[applied={T|T < 7315684778957299712}]
I20260806 22:37:34.238495 21507 catalog_manager.cc:1347] Generated new cluster ID: f72974a4e0394b438f727adc57676e99
I20260806 22:37:34.238551 21507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:37:34 dist-test-slave-3kk6 krb5kdc[21378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:34.246734 21496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:34.233675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.16.235.126:38701)
0806 22:37:34.233952 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:34.233963 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:34.234002 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:37:34.234129 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:34.234136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:34.234197 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:34.234371 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:34.235292 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:34.236982 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:34.238147 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:34.238779 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:34.238895 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:34.242047 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:34.242051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:34.242051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:34.242065 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:34.244895 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:34.245355 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:34.245358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:34.245359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:34.245475 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:34.245704 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:34.245704 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:34.245705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:34.246042 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:34.246192 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:34.246316 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 22:37:34.246465 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":139,"threads_started":1}
I20260806 22:37:34.247947 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:37:34.248098 21507 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:34.248780 21507 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:34.254997 21507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ecfb4a489eb42e5a37631abad9429af: Generated new TSK 0
I20260806 22:37:34.255206 21507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:37:34.266572 21496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:34.262483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.16.235.126:38701)
0806 22:37:34.262552 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:34.262555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:34.262699 (+ 144us) server_negotiation.cc:407] Connection header received
0806 22:37:34.262827 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:34.262829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:34.262842 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:34.262905 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:34.263288 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:34.264109 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:34.264860 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:34.265223 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:34.265301 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:34.266043 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:34.266044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:34.266044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:34.266305 (+ 261us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:34.266326 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:34.266333 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:34.266393 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:34.266450 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:37:34.266482 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:37:34.269469 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 21395
2026-08-06T22:37:34Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1565 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:34 dist-test-slave-3kk6 krb5kdc[21537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:34 dist-test-slave-3kk6 krb5kdc[21537](info): set up 2 sockets
Aug 06 22:37:34 dist-test-slave-3kk6 krb5kdc[21537](info): commencing operation
krb5kdc: starting...
W20260806 22:37:35.492800 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:35 dist-test-slave-3kk6 krb5kdc[21537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:35.629187 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37703
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:37703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:35.718966 21556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:35.719269 21556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:35.719313 21556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:35.720641 21556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:35.720747 21556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:35.720774 21556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:35.720798 21556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:35.720824 21556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:35.722545 21556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:37703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:35.722941 21556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:35.723214 21556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:35.728669 21566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:35.728616 21562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:35.728616 21569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:35.729036 21556 server_base.cc:1034] running on GCE node
I20260806 22:37:35.729341 21556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:35.729672 21556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:35.730845 21556 hybrid_clock.cc:648] HybridClock initialized: now 1786055855730817 us; error 51 us; skew 500 ppm
Aug 06 22:37:35 dist-test-slave-3kk6 krb5kdc[21537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055855, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:35.738483 21556 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:35.739145 21556 webserver.cc:460] Webserver started at http://127.16.235.126:34227/ using document root <none> and password file <none>
I20260806 22:37:35.739411 21556 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:35.739486 21556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:35.739651 21556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:35.740854 21556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4"
format_stamp: "Formatted at 2026-08-06 22:37:35 on dist-test-slave-3kk6"
server_key: "a135da5f27d4d3342ba3be48ea81046d"
server_key_iv: "5a044c3504175c634ac8f7b1865d8b8f"
server_key_version: "encryptionkey@0"
I20260806 22:37:35.741264 21556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4"
format_stamp: "Formatted at 2026-08-06 22:37:35 on dist-test-slave-3kk6"
server_key: "a135da5f27d4d3342ba3be48ea81046d"
server_key_iv: "5a044c3504175c634ac8f7b1865d8b8f"
server_key_version: "encryptionkey@0"
I20260806 22:37:35.742776 21556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:35.743600 21579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:35.743811 21556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:35.743932 21556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4"
format_stamp: "Formatted at 2026-08-06 22:37:35 on dist-test-slave-3kk6"
server_key: "a135da5f27d4d3342ba3be48ea81046d"
server_key_iv: "5a044c3504175c634ac8f7b1865d8b8f"
server_key_version: "encryptionkey@0"
I20260806 22:37:35.744016 21556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:35.757458 21556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:35.759819 21556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:35.760017 21556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:35.764482 21556 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:37703
I20260806 22:37:35.764564 21649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:37703 every 8 connection(s)
I20260806 22:37:35.765069 21556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:35.765779 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 21556
I20260806 22:37:35.765872 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:35.766074 17325 external_mini_cluster.cc:1428] Setting key 8b1ff0750dfef91e01899462c0ab2e47
I20260806 22:37:35.766486 21650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:35.770152 21650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4: Bootstrap starting.
I20260806 22:37:35.770946 21650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:35.771287 21650 log.cc:824] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:35.772132 21650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4: No bootstrap required, opened a new log
Aug 06 22:37:35 dist-test-slave-3kk6 krb5kdc[21537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:35.774880 21650 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37703 } }
I20260806 22:37:35.775074 21650 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:35.775121 21650 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1272e8cb9c64b50bc7fe9f72b0b69b4, State: Initialized, Role: FOLLOWER
I20260806 22:37:35.775274 21650 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37703 } }
I20260806 22:37:35.775341 21650 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:35.775364 21650 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:35.775434 21650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:35.776031 21650 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37703 } }
I20260806 22:37:35.776124 21650 leader_election.cc:302] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1272e8cb9c64b50bc7fe9f72b0b69b4; no voters:
I20260806 22:37:35.776386 21650 leader_election.cc:288] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:35.776420 21658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:35.776542 21658 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [term 1 LEADER]: Becoming Leader. State: Replica: e1272e8cb9c64b50bc7fe9f72b0b69b4, State: Running, Role: LEADER
I20260806 22:37:35.776633 21658 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37703 } }
I20260806 22:37:35.776695 21650 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:35.777012 21658 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1272e8cb9c64b50bc7fe9f72b0b69b4. Latest consensus state: current_term: 1 leader_uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37703 } } }
I20260806 22:37:35.777091 21658 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:35.777031 21659 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1272e8cb9c64b50bc7fe9f72b0b69b4" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37703 } } }
I20260806 22:37:35.777261 21659 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:35.777793 21666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:35.778927 21666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:35.779136 21656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:35.766727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47340 (local address 127.16.235.126:37703)
0806 22:37:35.766940 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:35.766950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:35.766986 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:37:35.767757 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:35.767768 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:35.767835 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:35.768003 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:35.768639 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:35.770052 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:35.770981 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:35.771612 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:35.771728 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:35.774547 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:35.774551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:35.774551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:35.774566 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:35.776940 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:35.777451 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:35.777454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:35.777454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:35.777562 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:35.777846 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:35.777847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:35.777847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:35.778203 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:35.778382 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:35.778787 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 22:37:35.778930 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260806 22:37:35.780833 21666 catalog_manager.cc:1347] Generated new cluster ID: 4affa5289f39448492f0ab62020f76a0
I20260806 22:37:35.780897 21666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:35.795631 21666 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:35.796366 21666 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:35.806030 21666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1272e8cb9c64b50bc7fe9f72b0b69b4: Generated new TSK 0
I20260806 22:37:35.806340 21666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:35.842108 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37703
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:35.942319 21682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:35.942564 21682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:35.942590 21682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:35.943889 21682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:35.943950 21682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:35.943981 21682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:35.945595 21682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:35.945907 21682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:35.946208 21682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:35.951860 21693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:35.951877 21690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:35.951853 21689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:35.952162 21682 server_base.cc:1034] running on GCE node
I20260806 22:37:35.952461 21682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:35.952772 21682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:35.953974 21682 hybrid_clock.cc:648] HybridClock initialized: now 1786055855953956 us; error 51 us; skew 500 ppm
Aug 06 22:37:35 dist-test-slave-3kk6 krb5kdc[21537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055855, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:35.957192 21682 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:35.957752 21682 webserver.cc:460] Webserver started at http://127.16.235.65:43765/ using document root <none> and password file <none>
I20260806 22:37:35.958038 21682 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:35.958118 21682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:35.958302 21682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:35.959758 21682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "ac9b131b3ecd41108c44f6acdca6e839"
format_stamp: "Formatted at 2026-08-06 22:37:35 on dist-test-slave-3kk6"
server_key: "102aa650a4291bf444e6d22da1418e86"
server_key_iv: "d2aeb3e2f9e03a7d4ac4b1afe37edb78"
server_key_version: "encryptionkey@0"
I20260806 22:37:35.960202 21682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "ac9b131b3ecd41108c44f6acdca6e839"
format_stamp: "Formatted at 2026-08-06 22:37:35 on dist-test-slave-3kk6"
server_key: "102aa650a4291bf444e6d22da1418e86"
server_key_iv: "d2aeb3e2f9e03a7d4ac4b1afe37edb78"
server_key_version: "encryptionkey@0"
I20260806 22:37:35.961853 21682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:35.962893 21701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:35.963121 21682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:35.963266 21682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "ac9b131b3ecd41108c44f6acdca6e839"
format_stamp: "Formatted at 2026-08-06 22:37:35 on dist-test-slave-3kk6"
server_key: "102aa650a4291bf444e6d22da1418e86"
server_key_iv: "d2aeb3e2f9e03a7d4ac4b1afe37edb78"
server_key_version: "encryptionkey@0"
I20260806 22:37:35.963358 21682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:35.974040 21682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:35.976949 21682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:35.977187 21682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:35.977548 21682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:35.977963 21682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:35.978022 21682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:35.978070 21682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:35.978113 21682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:35.984738 21682 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:34895
I20260806 22:37:35.984839 21858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:34895 every 8 connection(s)
I20260806 22:37:35.985229 21682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:35.989786 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 21682
I20260806 22:37:35.989946 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:35.990196 17325 external_mini_cluster.cc:1428] Setting key 3a008c7a8e0331de6eccf8078b6ba4ac
Aug 06 22:37:35 dist-test-slave-3kk6 krb5kdc[21537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055855, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:35.996268 21656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:35.986199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:32965 (local address 127.16.235.126:37703)
0806 22:37:35.986287 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:35.986290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:35.986515 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:37:35.987095 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:35.987097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:35.987108 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:35.987177 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:35.988258 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:35.988805 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:35.989952 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:35.990219 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:35.990321 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:35.992956 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:35.992958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:35.992958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:35.992967 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:35.994576 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:35.995125 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:35.995127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:35.995127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:35.995203 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:35.995490 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:35.995491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:35.995491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:35.995622 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:35.995704 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:35.996055 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 22:37:35.996157 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:37:35.996469 21863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:35.986135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37703 (local address 127.16.235.65:32965)
0806 22:37:35.986331 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:35.986379 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:35.986992 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:35.987269 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:35.987273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:35.987909 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:35.988126 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:35.988131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:35.988928 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:35.988930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:35.989819 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:35.989822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:35.990395 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:35.990396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:35.990518 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:35.991259 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:35.991272 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:35.992835 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:35.994755 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:35.994765 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:35.994768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:35.995035 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:35.995280 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:35.995280 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:35.995281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:35.995407 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:35.995697 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:35.995706 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:35.995904 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 22:37:35.996017 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 22:37:35.996202 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
I20260806 22:37:35.997037 21859 heartbeater.cc:347] Connected to a master server at 127.16.235.126:37703
I20260806 22:37:35.997146 21859 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:35.997373 21859 heartbeater.cc:511] Master 127.16.235.126:37703 requested a full tablet report, sending...
I20260806 22:37:35.998034 21599 ts_manager.cc:194] Registered new tserver with Master: ac9b131b3ecd41108c44f6acdca6e839 (127.16.235.65:34895)
I20260806 22:37:35.999043 21599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:32965
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:36.022840 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37703
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:36.115453 21868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:36.115700 21868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:36.115727 21868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:36.116938 21868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:36.116992 21868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:36.117022 21868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:36.118556 21868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:36.118887 21868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:36.119119 21868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:36.124441 21874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:36.124457 21877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:36.124588 21875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:36.124861 21868 server_base.cc:1034] running on GCE node
I20260806 22:37:36.125116 21868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:36.125378 21868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:36.126536 21868 hybrid_clock.cc:648] HybridClock initialized: now 1786055856126523 us; error 37 us; skew 500 ppm
Aug 06 22:37:36 dist-test-slave-3kk6 krb5kdc[21537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055856, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:36.129597 21868 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:36.130407 21868 webserver.cc:460] Webserver started at http://127.16.235.66:34529/ using document root <none> and password file <none>
I20260806 22:37:36.130779 21868 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:36.130856 21868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:36.131021 21868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:36.132241 21868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "ef70167bdcef4f0b9676bc27b3c38d56"
format_stamp: "Formatted at 2026-08-06 22:37:36 on dist-test-slave-3kk6"
server_key: "68116daaf9fd69341400256f9ccb0292"
server_key_iv: "7ca20e279db3ab9d9931cfdd36397e95"
server_key_version: "encryptionkey@0"
I20260806 22:37:36.132637 21868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "ef70167bdcef4f0b9676bc27b3c38d56"
format_stamp: "Formatted at 2026-08-06 22:37:36 on dist-test-slave-3kk6"
server_key: "68116daaf9fd69341400256f9ccb0292"
server_key_iv: "7ca20e279db3ab9d9931cfdd36397e95"
server_key_version: "encryptionkey@0"
I20260806 22:37:36.134174 21868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 22:37:36.135170 21890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:36.135421 21868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:37:36.135505 21868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "ef70167bdcef4f0b9676bc27b3c38d56"
format_stamp: "Formatted at 2026-08-06 22:37:36 on dist-test-slave-3kk6"
server_key: "68116daaf9fd69341400256f9ccb0292"
server_key_iv: "7ca20e279db3ab9d9931cfdd36397e95"
server_key_version: "encryptionkey@0"
I20260806 22:37:36.135601 21868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:36.149354 21868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:36.151944 21868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:36.152134 21868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:36.152422 21868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:36.152809 21868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:36.152851 21868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:36.152876 21868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:36.152925 21868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:36.159984 21868 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:42019
I20260806 22:37:36.160027 22037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:42019 every 8 connection(s)
I20260806 22:37:36.160418 21868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:37:36 dist-test-slave-3kk6 krb5kdc[21537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055856, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:36.169625 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 21868
I20260806 22:37:36.169811 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:36.170037 17325 external_mini_cluster.cc:1428] Setting key 423b4780d3d7431e3e2a0f45b6e128b8
I20260806 22:37:36.171427 21656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:36.161347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:48419 (local address 127.16.235.126:37703)
0806 22:37:36.161433 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:36.161436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:36.161737 (+ 301us) server_negotiation.cc:407] Connection header received
0806 22:37:36.162256 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:36.162257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:36.162269 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:36.162336 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:36.163448 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.163957 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.165027 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.165233 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.165305 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:36.167910 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:36.167912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:36.167912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:36.167920 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:36.169507 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:36.170131 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:36.170133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:36.170133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:36.170224 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:36.170597 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:36.170598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:36.170598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:36.170778 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:36.170858 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:36.171266 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 22:37:36.171358 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:37:36.171775 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:36.161336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37703 (local address 127.16.235.66:48419)
0806 22:37:36.161596 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:36.161640 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:36.162166 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:36.162395 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:36.162398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:36.163113 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:36.163337 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:36.163344 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.164057 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.164058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:36.164937 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:36.164940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.165321 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.165322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:36.165438 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:36.166073 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:36.166082 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:36.167794 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:36.169613 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:36.169622 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:36.169624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:36.170038 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:36.170346 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:36.170348 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:36.170349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:36.170524 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:36.170859 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:36.170868 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:36.171092 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 22:37:36.171232 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 22:37:36.171448 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":129,"threads_started":1}
I20260806 22:37:36.172412 22040 heartbeater.cc:347] Connected to a master server at 127.16.235.126:37703
I20260806 22:37:36.172536 22040 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:36.172758 22040 heartbeater.cc:511] Master 127.16.235.126:37703 requested a full tablet report, sending...
I20260806 22:37:36.173281 21599 ts_manager.cc:194] Registered new tserver with Master: ef70167bdcef4f0b9676bc27b3c38d56 (127.16.235.66:42019)
I20260806 22:37:36.173836 21599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:48419
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:36.202503 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37703
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:36.290928 22046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:36.291172 22046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:36.291198 22046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:36.292507 22046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:36.292564 22046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:36.292593 22046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:36.294142 22046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:36.294425 22046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:36.294749 22046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:36.300199 22055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:36.300209 22053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:36.300508 22057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:36.300515 22046 server_base.cc:1034] running on GCE node
I20260806 22:37:36.300822 22046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:36.301088 22046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:36.302284 22046 hybrid_clock.cc:648] HybridClock initialized: now 1786055856302269 us; error 33 us; skew 500 ppm
Aug 06 22:37:36 dist-test-slave-3kk6 krb5kdc[21537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055856, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:36.305158 22046 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:36.305693 22046 webserver.cc:460] Webserver started at http://127.16.235.67:41447/ using document root <none> and password file <none>
I20260806 22:37:36.305974 22046 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:36.306025 22046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:36.306174 22046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:36.307401 22046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "0ea5a38dc027402fa0e86a86755468e3"
format_stamp: "Formatted at 2026-08-06 22:37:36 on dist-test-slave-3kk6"
server_key: "de778f8af936b2dc299b631a2dc64e1f"
server_key_iv: "17f74ad4f5230d6cf8065e0e43b1ed50"
server_key_version: "encryptionkey@0"
I20260806 22:37:36.307873 22046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "0ea5a38dc027402fa0e86a86755468e3"
format_stamp: "Formatted at 2026-08-06 22:37:36 on dist-test-slave-3kk6"
server_key: "de778f8af936b2dc299b631a2dc64e1f"
server_key_iv: "17f74ad4f5230d6cf8065e0e43b1ed50"
server_key_version: "encryptionkey@0"
I20260806 22:37:36.309374 22046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:37:36.310196 22069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:36.310410 22046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:36.310524 22046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "0ea5a38dc027402fa0e86a86755468e3"
format_stamp: "Formatted at 2026-08-06 22:37:36 on dist-test-slave-3kk6"
server_key: "de778f8af936b2dc299b631a2dc64e1f"
server_key_iv: "17f74ad4f5230d6cf8065e0e43b1ed50"
server_key_version: "encryptionkey@0"
I20260806 22:37:36.310608 22046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:36.330403 22046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:36.333088 22046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:36.333307 22046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:36.333597 22046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:36.333971 22046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:36.334031 22046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:36.334075 22046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:36.334106 22046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:36.340896 22046 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:35935
I20260806 22:37:36.340945 22225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:35935 every 8 connection(s)
I20260806 22:37:36.341322 22046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:37:36 dist-test-slave-3kk6 krb5kdc[21537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055856, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:36.349628 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 22046
I20260806 22:37:36.349740 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:36.349977 17325 external_mini_cluster.cc:1428] Setting key f45da5a0d31c98f603b1493007ec6435
I20260806 22:37:36.352207 21656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:36.342257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:44611 (local address 127.16.235.126:37703)
0806 22:37:36.342332 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:36.342336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:36.342409 (+ 73us) server_negotiation.cc:407] Connection header received
0806 22:37:36.343000 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:36.343001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:36.343011 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:36.343083 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:36.344101 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.344614 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.345754 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.345947 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.346008 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:36.348759 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:36.348760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:36.348760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:36.348768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:36.350713 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:36.351251 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:36.351251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:36.351252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:36.351311 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:36.351608 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:36.351609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:36.351609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:36.351730 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:36.351790 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:36.352072 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 22:37:36.352140 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:37:36.352502 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:36.342094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37703 (local address 127.16.235.67:44611)
0806 22:37:36.342275 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:36.342322 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:36.342903 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:36.343143 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:36.343147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:36.343770 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:36.343978 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:36.343983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.344785 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.344786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:36.345649 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:36.345652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.346058 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.346060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:36.346199 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:36.346870 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:36.346880 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:36.348626 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:36.350799 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:36.350815 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:36.350817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:36.351174 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:36.351407 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:36.351408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:36.351409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:36.351541 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:36.351799 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:36.351806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:36.351946 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 22:37:36.352047 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:37:36.352211 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260806 22:37:36.352988 22226 heartbeater.cc:347] Connected to a master server at 127.16.235.126:37703
I20260806 22:37:36.353108 22226 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:36.353315 22226 heartbeater.cc:511] Master 127.16.235.126:37703 requested a full tablet report, sending...
I20260806 22:37:36.353826 21599 ts_manager.cc:194] Registered new tserver with Master: 0ea5a38dc027402fa0e86a86755468e3 (127.16.235.67:35935)
I20260806 22:37:36.354274 21599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:44611
I20260806 22:37:36.362033 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:37:36 dist-test-slave-3kk6 krb5kdc[21537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055856, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 22:37:36 dist-test-slave-3kk6 krb5kdc[21537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055856, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:36.388162 21656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:36.380442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.16.235.126:37703)
0806 22:37:36.380515 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:36.380517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:36.380623 (+ 106us) server_negotiation.cc:407] Connection header received
0806 22:37:36.380713 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:36.380714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:36.380724 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:36.380782 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:36.381483 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.381998 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.382853 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:36.383049 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:36.383109 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:36.385668 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:36.385670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:36.385670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:36.385678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:36.387099 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:36.387447 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:36.387448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:36.387448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:36.387502 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:36.387667 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:36.387668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:36.387668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:36.387826 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:36.387917 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:36.388021 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:37:36.388088 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 22:37:36.388476 21599 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47350
I20260806 22:37:36.389004 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 21682
I20260806 22:37:36.407972 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 21868
I20260806 22:37:36.424264 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 22046
I20260806 22:37:36.435051 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 21556
2026-08-06T22:37:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2166 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:36 dist-test-slave-3kk6 krb5kdc[22241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:36 dist-test-slave-3kk6 krb5kdc[22241](info): set up 2 sockets
Aug 06 22:37:36 dist-test-slave-3kk6 krb5kdc[22241](info): commencing operation
krb5kdc: starting...
W20260806 22:37:37.662947 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:37 dist-test-slave-3kk6 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:37.797355 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34431
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:37.882603 22256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:37.882925 22256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:37.882967 22256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:37.884187 22256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:37.884276 22256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:37.884299 22256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:37.884321 22256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:37.884353 22256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:37.886024 22256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:34431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:37.886332 22256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:37.886587 22256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:37.891717 22263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:37.891737 22268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:37.891764 22262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:37.891997 22256 server_base.cc:1034] running on GCE node
I20260806 22:37:37.892258 22256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:37.892578 22256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:37.893765 22256 hybrid_clock.cc:648] HybridClock initialized: now 1786055857893743 us; error 44 us; skew 500 ppm
Aug 06 22:37:37 dist-test-slave-3kk6 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:37.896672 22256 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:37.897219 22256 webserver.cc:460] Webserver started at http://127.16.235.126:34321/ using document root <none> and password file <none>
I20260806 22:37:37.897480 22256 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:37.897553 22256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:37.897713 22256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:37.898928 22256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "1367327fdeb04b549be288e3644e7bb3"
format_stamp: "Formatted at 2026-08-06 22:37:37 on dist-test-slave-3kk6"
server_key: "7fc1c0bea5526a6379afe97eaaff83db"
server_key_iv: "d6eccf7463fb8a2940c6fc47cfaa95bc"
server_key_version: "encryptionkey@0"
I20260806 22:37:37.899324 22256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "1367327fdeb04b549be288e3644e7bb3"
format_stamp: "Formatted at 2026-08-06 22:37:37 on dist-test-slave-3kk6"
server_key: "7fc1c0bea5526a6379afe97eaaff83db"
server_key_iv: "d6eccf7463fb8a2940c6fc47cfaa95bc"
server_key_version: "encryptionkey@0"
I20260806 22:37:37.900856 22256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:37.901607 22277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:37.901815 22256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:37.901916 22256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "1367327fdeb04b549be288e3644e7bb3"
format_stamp: "Formatted at 2026-08-06 22:37:37 on dist-test-slave-3kk6"
server_key: "7fc1c0bea5526a6379afe97eaaff83db"
server_key_iv: "d6eccf7463fb8a2940c6fc47cfaa95bc"
server_key_version: "encryptionkey@0"
I20260806 22:37:37.902004 22256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:37.912905 22256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:37.915186 22256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:37.915367 22256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:37.920022 22256 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:34431
I20260806 22:37:37.920081 22354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:34431 every 8 connection(s)
I20260806 22:37:37.920650 22256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:37.921378 22355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:37.923403 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 22256
I20260806 22:37:37.923509 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:37.923702 17325 external_mini_cluster.cc:1428] Setting key 55ebea948f7840495385c35480d5a9f1
I20260806 22:37:37.924199 22355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3: Bootstrap starting.
I20260806 22:37:37.924901 22355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:37.925246 22355 log.cc:824] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:37.928504 22355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3: No bootstrap required, opened a new log
Aug 06 22:37:37 dist-test-slave-3kk6 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:37.931378 22355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1367327fdeb04b549be288e3644e7bb3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34431 } }
I20260806 22:37:37.931555 22355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:37.931605 22355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1367327fdeb04b549be288e3644e7bb3, State: Initialized, Role: FOLLOWER
I20260806 22:37:37.931787 22355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1367327fdeb04b549be288e3644e7bb3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34431 } }
I20260806 22:37:37.931886 22355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:37.931934 22355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:37.931984 22355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:37.932592 22355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1367327fdeb04b549be288e3644e7bb3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34431 } }
I20260806 22:37:37.932720 22355 leader_election.cc:302] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1367327fdeb04b549be288e3644e7bb3; no voters:
I20260806 22:37:37.932945 22355 leader_election.cc:288] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:37.933017 22361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:37.933193 22361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [term 1 LEADER]: Becoming Leader. State: Replica: 1367327fdeb04b549be288e3644e7bb3, State: Running, Role: LEADER
I20260806 22:37:37.933245 22355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:37.933326 22361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1367327fdeb04b549be288e3644e7bb3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34431 } }
I20260806 22:37:37.934226 22364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1367327fdeb04b549be288e3644e7bb3. Latest consensus state: current_term: 1 leader_uuid: "1367327fdeb04b549be288e3644e7bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1367327fdeb04b549be288e3644e7bb3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34431 } } }
I20260806 22:37:37.934345 22364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:37.934876 22362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1367327fdeb04b549be288e3644e7bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1367327fdeb04b549be288e3644e7bb3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34431 } } }
I20260806 22:37:37.935144 22362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:37.935840 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:37.924317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.16.235.126:34431)
0806 22:37:37.924534 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:37.924546 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:37.924579 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:37:37.925083 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:37.925091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:37.925141 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:37.925266 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:37.926098 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:37.927441 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:37.928202 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:37.928656 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:37.928757 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:37.931569 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:37.931573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:37.931573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:37.931604 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:37.933857 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:37.934281 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:37.934284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:37.934284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:37.934397 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:37.934637 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:37.934638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:37.934638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:37.935012 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:37.935171 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:37.935482 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 22:37:37.935635 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":118,"threads_started":1}
W20260806 22:37:37.935926 22381 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:37:37.935976 22381 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:37:37.936028 22371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:37.936215 22371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:37.937718 22371 catalog_manager.cc:1347] Generated new cluster ID: c81fd88ee4944740a003cfa05bfe772e
I20260806 22:37:37.937795 22371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:37.941658 22371 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:37.942205 22371 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:37.949467 22371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1367327fdeb04b549be288e3644e7bb3: Generated new TSK 0
I20260806 22:37:37.949676 22371 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:37.979693 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34431
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:38.066207 22387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:38.066437 22387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:38.066462 22387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:38.067679 22387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:38.067728 22387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:38.067756 22387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:38.069250 22387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:38.069450 22387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:38.069654 22387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:38.075022 22394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:38.075026 22393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:38.075210 22387 server_base.cc:1034] running on GCE node
W20260806 22:37:38.075078 22397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:38.075525 22387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:38.075750 22387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:38.076920 22387 hybrid_clock.cc:648] HybridClock initialized: now 1786055858076892 us; error 51 us; skew 500 ppm
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:38.079671 22387 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:38.080144 22387 webserver.cc:460] Webserver started at http://127.16.235.65:38287/ using document root <none> and password file <none>
I20260806 22:37:38.080380 22387 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:38.080454 22387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:38.080614 22387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:38.081830 22387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "bf39863e4fa74da5aaefb022246f5fc0"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-3kk6"
server_key: "bf8beaa6aa3a80ef1591384f47af9a4a"
server_key_iv: "0002362a6e14d5f3cf616ed30fadfa40"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.082216 22387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "bf39863e4fa74da5aaefb022246f5fc0"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-3kk6"
server_key: "bf8beaa6aa3a80ef1591384f47af9a4a"
server_key_iv: "0002362a6e14d5f3cf616ed30fadfa40"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.083801 22387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:38.084560 22407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.084753 22387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:38.084869 22387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "bf39863e4fa74da5aaefb022246f5fc0"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-3kk6"
server_key: "bf8beaa6aa3a80ef1591384f47af9a4a"
server_key_iv: "0002362a6e14d5f3cf616ed30fadfa40"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.084949 22387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:38.099080 22387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:38.102427 22387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:38.102605 22387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:38.102914 22387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:38.103256 22387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:38.103322 22387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.103366 22387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:38.103394 22387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.109622 22387 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:46457
I20260806 22:37:38.109702 22566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:46457 every 8 connection(s)
I20260806 22:37:38.110069 22387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:38.115999 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 22387
I20260806 22:37:38.116130 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:38.116336 17325 external_mini_cluster.cc:1428] Setting key 95a1c08c8010aac53fbb12656d85b060
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:38.120240 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:38.110893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34431 (local address 127.16.235.65:56557)
0806 22:37:38.111080 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:38.111122 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:38.112079 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:38.112336 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:38.112340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:38.112604 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:38.112800 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:38.112804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.113531 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.113531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:38.114345 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:38.114347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.114765 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.114766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:38.114866 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:38.115351 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:38.115359 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:38.117051 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:38.118851 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:38.118860 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:38.118863 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:38.119113 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:38.119340 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:38.119341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:38.119341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:38.119444 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:38.119726 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:38.119731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:38.119854 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 22:37:38.119940 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:37:38.120068 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260806 22:37:38.120247 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:38.110934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:56557 (local address 127.16.235.126:34431)
0806 22:37:38.110980 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:38.110982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:38.111243 (+ 261us) server_negotiation.cc:407] Connection header received
0806 22:37:38.112200 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:38.112201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:38.112214 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:38.112282 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:38.112915 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.113437 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.114422 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.114623 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.114755 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:38.117162 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:38.117163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:38.117164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:38.117170 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:38.118749 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.119190 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.119191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.119192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.119270 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.119508 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.119509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.119509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.119658 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:38.119728 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:38.119989 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 22:37:38.120099 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:37:38.120795 22567 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34431
I20260806 22:37:38.120925 22567 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:38.121122 22567 heartbeater.cc:511] Master 127.16.235.126:34431 requested a full tablet report, sending...
I20260806 22:37:38.121765 22299 ts_manager.cc:194] Registered new tserver with Master: bf39863e4fa74da5aaefb022246f5fc0 (127.16.235.65:46457)
I20260806 22:37:38.122792 22299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:56557
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:38.143433 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34431
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:38.228790 22575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:38.229086 22575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:38.229135 22575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:38.230765 22575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:38.230846 22575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:38.230892 22575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:38.232820 22575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:38.233031 22575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:38.233304 22575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:38.238613 22590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:38.238575 22588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:38.238566 22584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:38.238873 22575 server_base.cc:1034] running on GCE node
I20260806 22:37:38.239148 22575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:38.239373 22575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:38.240550 22575 hybrid_clock.cc:648] HybridClock initialized: now 1786055858240536 us; error 31 us; skew 500 ppm
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:38.243319 22575 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:38.243806 22575 webserver.cc:460] Webserver started at http://127.16.235.66:43885/ using document root <none> and password file <none>
I20260806 22:37:38.244055 22575 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:38.244102 22575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:38.244278 22575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:38.245352 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "16dba4eb5d054a7fab8b976e38955f51"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-3kk6"
server_key: "dacadd2a4f1ef74133b145c7b6aa2f68"
server_key_iv: "741e6358c1bb132439ee261c618776ce"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.245738 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "16dba4eb5d054a7fab8b976e38955f51"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-3kk6"
server_key: "dacadd2a4f1ef74133b145c7b6aa2f68"
server_key_iv: "741e6358c1bb132439ee261c618776ce"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.247175 22575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:38.247881 22601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.248059 22575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:38.248149 22575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "16dba4eb5d054a7fab8b976e38955f51"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-3kk6"
server_key: "dacadd2a4f1ef74133b145c7b6aa2f68"
server_key_iv: "741e6358c1bb132439ee261c618776ce"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.248224 22575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:38.261929 22575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:38.264478 22575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:38.264659 22575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:38.264914 22575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:38.265246 22575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:38.265301 22575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.265345 22575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:38.265373 22575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.271751 22575 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:37461
I20260806 22:37:38.271793 22764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:37461 every 8 connection(s)
I20260806 22:37:38.272158 22575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:38.279359 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 22575
I20260806 22:37:38.279457 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:38.279645 17325 external_mini_cluster.cc:1428] Setting key f0e0f7006534dd6b199b6fed9c800542
I20260806 22:37:38.283776 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:38.272915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:42203 (local address 127.16.235.126:34431)
0806 22:37:38.272998 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:38.273000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:38.273179 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:37:38.274118 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:38.274119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:38.274130 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:38.274180 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:38.274854 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.275353 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.276346 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.276532 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.276585 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:38.278751 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:38.278752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:38.278753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:38.278759 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:38.280376 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.282755 (+ 2379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.282756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.282757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.282827 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.283098 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.283099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.283100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.283222 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:38.283302 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:38.283586 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 22:37:38.283674 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:37:38.283968 22768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:38.272862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34431 (local address 127.16.235.66:42203)
0806 22:37:38.273051 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:38.273088 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:38.274023 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:38.274243 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:38.274247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:38.274509 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:38.274766 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:38.274777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.275435 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.275436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:38.276262 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:38.276264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.276606 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.276607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:38.276707 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:38.277200 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:38.277209 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:38.278596 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:38.282372 (+ 3776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:38.282382 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:38.282385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:38.282641 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:38.282906 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:38.282906 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:38.282907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:38.283021 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:38.283295 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:38.283302 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:38.283445 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 22:37:38.283551 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:37:38.283718 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":107,"threads_started":1}
I20260806 22:37:38.284488 22766 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34431
I20260806 22:37:38.284577 22766 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:38.284813 22766 heartbeater.cc:511] Master 127.16.235.126:34431 requested a full tablet report, sending...
I20260806 22:37:38.285252 22299 ts_manager.cc:194] Registered new tserver with Master: 16dba4eb5d054a7fab8b976e38955f51 (127.16.235.66:37461)
I20260806 22:37:38.285645 22299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:42203
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:38.309012 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34431
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:38.393110 22773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:38.393374 22773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:38.393417 22773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:38.394598 22773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:38.394696 22773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:38.394759 22773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:38.396431 22773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:38.396705 22773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:38.396952 22773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:38.402323 22779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:38.402437 22773 server_base.cc:1034] running on GCE node
W20260806 22:37:38.402311 22781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:38.402293 22778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:38.402818 22773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:38.403092 22773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:38.404264 22773 hybrid_clock.cc:648] HybridClock initialized: now 1786055858404249 us; error 31 us; skew 500 ppm
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:38.407100 22773 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:38.407502 22773 webserver.cc:460] Webserver started at http://127.16.235.67:39609/ using document root <none> and password file <none>
I20260806 22:37:38.407685 22773 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:38.407725 22773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:38.407835 22773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:38.408867 22773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "d50f3a91d3194e9bb9312566f1e222c0"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-3kk6"
server_key: "c3400aee4499eb447dbf95082172d10b"
server_key_iv: "79faed0a2e2f939f014a9b08628cb0ec"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.409188 22773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "d50f3a91d3194e9bb9312566f1e222c0"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-3kk6"
server_key: "c3400aee4499eb447dbf95082172d10b"
server_key_iv: "79faed0a2e2f939f014a9b08628cb0ec"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.410483 22773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:38.411279 22792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.411443 22773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:37:38.411564 22773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "d50f3a91d3194e9bb9312566f1e222c0"
format_stamp: "Formatted at 2026-08-06 22:37:38 on dist-test-slave-3kk6"
server_key: "c3400aee4499eb447dbf95082172d10b"
server_key_iv: "79faed0a2e2f939f014a9b08628cb0ec"
server_key_version: "encryptionkey@0"
I20260806 22:37:38.411645 22773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:38.421808 22773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:38.423818 22773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:38.423938 22773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:38.424139 22773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:38.424396 22773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:38.424427 22773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.424449 22773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:38.424463 22773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:38.430377 22773 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:40185
I20260806 22:37:38.430475 22948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:40185 every 8 connection(s)
I20260806 22:37:38.430845 22773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:38.434886 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 22773
I20260806 22:37:38.435015 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:38.435249 17325 external_mini_cluster.cc:1428] Setting key e96a20c46eb3c16e5795bf220b58fb21
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:38.441207 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:38.431590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:36615 (local address 127.16.235.126:34431)
0806 22:37:38.431640 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:38.431642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:38.431875 (+ 233us) server_negotiation.cc:407] Connection header received
0806 22:37:38.432895 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:38.432896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:38.432906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:38.432968 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:38.433561 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.434161 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.435617 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.435935 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.436001 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:38.438226 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:38.438227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:38.438228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:38.438236 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:38.439733 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.440176 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.440177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.440178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.440239 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.440507 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.440508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.440508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.440629 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:38.440706 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:38.441027 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 22:37:38.441109 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:37:38.441431 22951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:38.431557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34431 (local address 127.16.235.67:36615)
0806 22:37:38.431738 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:38.431782 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:38.432798 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:38.433022 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:38.433026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:38.433290 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:38.433488 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:38.433492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.434261 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.434262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:38.435543 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:38.435546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.436023 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.436024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:38.436159 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:38.436735 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:38.436744 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:38.438117 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:38.439838 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:38.439846 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:38.439848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:38.440083 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:38.440314 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:38.440315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:38.440315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:38.440432 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:38.440697 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:38.440704 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:38.440888 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 22:37:38.441004 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:37:38.441175 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":122,"threads_started":1}
I20260806 22:37:38.441948 22949 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34431
I20260806 22:37:38.442049 22949 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:38.442209 22949 heartbeater.cc:511] Master 127.16.235.126:34431 requested a full tablet report, sending...
I20260806 22:37:38.442571 22299 ts_manager.cc:194] Registered new tserver with Master: d50f3a91d3194e9bb9312566f1e222c0 (127.16.235.67:40185)
I20260806 22:37:38.442987 22299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:36615
I20260806 22:37:38.447201 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:38.473050 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:38.464898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57860 (local address 127.16.235.65:46457)
0806 22:37:38.465116 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:38.465122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:38.465134 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:37:38.465165 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:38.465167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:38.465183 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:38.465257 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:38.465851 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.466577 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.467407 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.467807 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.467874 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:38.470331 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:38.470341 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:38.470341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:38.470349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:38.471982 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.472316 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.472320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.472320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.472379 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.472549 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.472549 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.472550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.472692 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:38.472825 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:38.472860 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 22:37:38.472927 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":106,"threads_started":1}
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:38.481213 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:38.474524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47998 (local address 127.16.235.126:34431)
0806 22:37:38.474595 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:38.474597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:38.474608 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:38.474676 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:38.474677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:38.474689 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:38.474748 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:38.475236 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.475740 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.476433 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:38.476631 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:38.476714 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:38.478954 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:38.478955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:38.478955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:38.478963 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:38.480185 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.480512 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.480513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.480513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.480564 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:38.480770 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:38.480770 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:38.480771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:38.480923 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:38.481002 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:38.481085 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:37:38.481147 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 22:37:38.481482 22299 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47998
I20260806 22:37:38.481843 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 22387
I20260806 22:37:38.493914 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 22575
I20260806 22:37:38.504487 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 22773
I20260806 22:37:38.520956 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 22256
2026-08-06T22:37:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2085 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22965](info): set up 2 sockets
Aug 06 22:37:38 dist-test-slave-3kk6 krb5kdc[22965](info): commencing operation
krb5kdc: starting...
W20260806 22:37:39.751884 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:39 dist-test-slave-3kk6 krb5kdc[22965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:39.885808 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:46115
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:46115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:39.972673 22984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:39.972980 22984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:39.973029 22984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:39.974347 22984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:39.974426 22984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:39.974452 22984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:39.974476 22984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:39.974509 22984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:39.976245 22984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:46115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:46115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:39.976583 22984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:39.976850 22984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:39.982223 22998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:39.982255 22990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:39.982569 22984 server_base.cc:1034] running on GCE node
W20260806 22:37:39.982223 22991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:39.982933 22984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:39.983299 22984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:39.984483 22984 hybrid_clock.cc:648] HybridClock initialized: now 1786055859984474 us; error 42 us; skew 500 ppm
Aug 06 22:37:39 dist-test-slave-3kk6 krb5kdc[22965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055859, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:39.987401 22984 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:39.987811 22984 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:39.987905 22984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:39.988075 22984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:39.989245 22984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd"
format_stamp: "Formatted at 2026-08-06 22:37:39 on dist-test-slave-3kk6"
server_key: "5fe78e551dec8e8811a5074f2e55783a"
server_key_iv: "527e8b29de8e00a7a853a707bd041207"
server_key_version: "encryptionkey@0"
I20260806 22:37:39.989665 22984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd"
format_stamp: "Formatted at 2026-08-06 22:37:39 on dist-test-slave-3kk6"
server_key: "5fe78e551dec8e8811a5074f2e55783a"
server_key_iv: "527e8b29de8e00a7a853a707bd041207"
server_key_version: "encryptionkey@0"
I20260806 22:37:39.991194 22984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:39.992054 23007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:39.992242 22984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:39.992367 22984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd"
format_stamp: "Formatted at 2026-08-06 22:37:39 on dist-test-slave-3kk6"
server_key: "5fe78e551dec8e8811a5074f2e55783a"
server_key_iv: "527e8b29de8e00a7a853a707bd041207"
server_key_version: "encryptionkey@0"
I20260806 22:37:39.992450 22984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:40.004540 22984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:40.006845 22984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:40.007040 22984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:40.011742 22984 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:46115
I20260806 22:37:40.011785 23080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:46115 every 8 connection(s)
I20260806 22:37:40.012110 22984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:40.012319 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 22984
I20260806 22:37:40.012460 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:40.012671 17325 external_mini_cluster.cc:1428] Setting key 75cda47f37c6a4a23b8f2d65047f5210
I20260806 22:37:40.013499 23083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:40.016247 23083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd: Bootstrap starting.
I20260806 22:37:40.016906 23083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:40.017194 23083 log.cc:824] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:40.017928 23083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd: No bootstrap required, opened a new log
Aug 06 22:37:40 dist-test-slave-3kk6 krb5kdc[22965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:40.019925 23083 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46115 } }
I20260806 22:37:40.020092 23083 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:40.020141 23083 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ff3fcafd3e4005a1eff2af806f7cfd, State: Initialized, Role: FOLLOWER
I20260806 22:37:40.020290 23083 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46115 } }
I20260806 22:37:40.020377 23083 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:40.020414 23083 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:40.020463 23083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:40.021020 23083 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46115 } }
I20260806 22:37:40.021142 23083 leader_election.cc:302] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7ff3fcafd3e4005a1eff2af806f7cfd; no voters:
I20260806 22:37:40.021333 23083 leader_election.cc:288] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:40.021366 23089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:40.021482 23089 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [term 1 LEADER]: Becoming Leader. State: Replica: f7ff3fcafd3e4005a1eff2af806f7cfd, State: Running, Role: LEADER
I20260806 22:37:40.021580 23089 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46115 } }
I20260806 22:37:40.021657 23083 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:40.021940 23089 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46115 } } }
I20260806 22:37:40.022089 23089 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:40.022284 23089 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7ff3fcafd3e4005a1eff2af806f7cfd. Latest consensus state: current_term: 1 leader_uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ff3fcafd3e4005a1eff2af806f7cfd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46115 } } }
I20260806 22:37:40.022418 23089 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:40.022819 23098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:40.023449 23098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:40.024438 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.013356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.16.235.126:46115)
0806 22:37:40.013590 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.013618 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.013655 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:37:40.014132 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.014139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.014185 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.014341 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:40.015089 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.016019 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.016881 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.017444 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.017565 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.020034 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:40.020038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:40.020039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:40.020052 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:40.022332 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.022799 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.022802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.022803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.022918 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.023152 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.023152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.023153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.023477 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:40.024011 (+ 534us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.024087 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.024232 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":102,"threads_started":1}
I20260806 22:37:40.025379 23098 catalog_manager.cc:1347] Generated new cluster ID: 5013db3107494aebbd7ca57390508a05
I20260806 22:37:40.025440 23098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:40.036597 23098 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:40.037166 23098 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:40.043510 23098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd: Generated new TSK 0
I20260806 22:37:40.043743 23098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:40.081856 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46115
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:40.174247 23123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:40.174554 23123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:40.174602 23123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:40.175884 23123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:40.175967 23123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:40.176015 23123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:40.177659 23123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:40.177963 23123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:40.178237 23123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:40.183588 23134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:40.183636 23129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:40.183840 23123 server_base.cc:1034] running on GCE node
W20260806 22:37:40.183588 23131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:40.184108 23123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:40.184412 23123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:40.185598 23123 hybrid_clock.cc:648] HybridClock initialized: now 1786055860185577 us; error 40 us; skew 500 ppm
Aug 06 22:37:40 dist-test-slave-3kk6 krb5kdc[22965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:40.188616 23123 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:40.189036 23123 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:40.189092 23123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:40.189200 23123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:40.190315 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "243ba7d90fce4218a3ff5deb5fffa5dd"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "75aa5c958b92ed70f584f65f7321d343"
server_key_iv: "826b41b34d11f57a042da2a6c9006dc1"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.190735 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "243ba7d90fce4218a3ff5deb5fffa5dd"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "75aa5c958b92ed70f584f65f7321d343"
server_key_iv: "826b41b34d11f57a042da2a6c9006dc1"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.192239 23123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:40.192986 23144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.193173 23123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.193251 23123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "243ba7d90fce4218a3ff5deb5fffa5dd"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "75aa5c958b92ed70f584f65f7321d343"
server_key_iv: "826b41b34d11f57a042da2a6c9006dc1"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.193310 23123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:40.201061 23123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:40.203657 23123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:40.203794 23123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:40.204020 23123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:40.204325 23123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:40.204360 23123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.204384 23123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:40.204397 23123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.211022 23123 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:38239
I20260806 22:37:40.211091 23299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:38239 every 8 connection(s)
I20260806 22:37:40.211403 23123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:37:40 dist-test-slave-3kk6 krb5kdc[22965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:40.218858 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 23123
I20260806 22:37:40.218988 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:40.219211 17325 external_mini_cluster.cc:1428] Setting key 5f8076bfa1b8c75adfaedc75590bf969
I20260806 22:37:40.222672 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.212301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:48099 (local address 127.16.235.126:46115)
0806 22:37:40.212378 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.212380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.212614 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:37:40.213104 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.213105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.213115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.213175 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:40.214142 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.214688 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.215747 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.215954 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.216012 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.218582 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:40.218584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:40.218585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:40.218594 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:40.220733 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.221222 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.221223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.221223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.221293 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.221658 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.221659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.221659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.221824 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:40.221902 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.222404 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.222493 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:37:40.222997 23305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.212284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:46115 (local address 127.16.235.65:48099)
0806 22:37:40.212468 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:40.212515 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:40.213019 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:40.213240 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:40.213243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:40.213521 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:40.214047 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.214052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.214820 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.214828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.215661 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.215664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.216055 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.216056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.216170 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.216931 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:40.216940 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:40.218449 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:40.220837 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:40.220846 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:40.220848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:40.221136 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:40.221386 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:40.221387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:40.221387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:40.221577 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:40.221932 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:40.221940 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:40.222176 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 22:37:40.222373 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 22:37:40.222608 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260806 22:37:40.223591 23300 heartbeater.cc:347] Connected to a master server at 127.16.235.126:46115
I20260806 22:37:40.223718 23300 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:40.223913 23300 heartbeater.cc:511] Master 127.16.235.126:46115 requested a full tablet report, sending...
I20260806 22:37:40.224581 23034 ts_manager.cc:194] Registered new tserver with Master: 243ba7d90fce4218a3ff5deb5fffa5dd (127.16.235.65:38239)
I20260806 22:37:40.225683 23034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:48099
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:40.249379 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46115
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:40.337607 23310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:40.337843 23310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:40.337867 23310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:40.339124 23310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:40.339169 23310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:40.339196 23310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:40.340768 23310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:40.341027 23310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:40.341284 23310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:40.346719 23325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:40.346707 23321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:40.346710 23320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:40.346999 23310 server_base.cc:1034] running on GCE node
I20260806 22:37:40.347218 23310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:40.347429 23310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:40.348604 23310 hybrid_clock.cc:648] HybridClock initialized: now 1786055860348599 us; error 58 us; skew 500 ppm
Aug 06 22:37:40 dist-test-slave-3kk6 krb5kdc[22965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:40.351418 23310 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:40.351897 23310 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:40.351977 23310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:40.352140 23310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:40.353271 23310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "7302a2eca14e4dd3ac13a59cd57880b8"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "56241a06c2cb5500313b2d8c37d99506"
server_key_iv: "07aa5d8390c099e667bb81e02bd5d86c"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.353679 23310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "7302a2eca14e4dd3ac13a59cd57880b8"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "56241a06c2cb5500313b2d8c37d99506"
server_key_iv: "07aa5d8390c099e667bb81e02bd5d86c"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.355185 23310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:40.356045 23336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.356228 23310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:40.356341 23310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "7302a2eca14e4dd3ac13a59cd57880b8"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "56241a06c2cb5500313b2d8c37d99506"
server_key_iv: "07aa5d8390c099e667bb81e02bd5d86c"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.356422 23310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:40.375629 23310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:40.378258 23310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:40.378445 23310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:40.378801 23310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:40.379160 23310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:40.379220 23310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.379266 23310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:40.379298 23310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.385576 23310 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:46233
I20260806 22:37:40.385622 23512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:46233 every 8 connection(s)
I20260806 22:37:40.385927 23310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:40.392315 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 23310
I20260806 22:37:40.392445 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
Aug 06 22:37:40 dist-test-slave-3kk6 krb5kdc[22965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:40.392655 17325 external_mini_cluster.cc:1428] Setting key 7c0e302ce8e17f2a1b1107a61df3bf2c
I20260806 22:37:40.396690 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.386716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:41217 (local address 127.16.235.126:46115)
0806 22:37:40.386773 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.386776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.387034 (+ 258us) server_negotiation.cc:407] Connection header received
0806 22:37:40.387591 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.387592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.387607 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.387662 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:40.388685 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.389190 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.390218 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.390421 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.390515 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.393193 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:40.393194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:40.393194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:40.393201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:40.394938 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.395389 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.395390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.395391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.395462 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.395742 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.395743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.395744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.395987 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:40.396105 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.396499 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.396592 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:37:40.396940 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.386714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:46115 (local address 127.16.235.66:41217)
0806 22:37:40.386904 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:40.386943 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:40.387474 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:40.387745 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:40.387749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:40.388043 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:40.388565 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.388571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.389310 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.389311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.390120 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.390122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.390528 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.390529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.390712 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.391278 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:40.391287 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:40.393100 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:40.395032 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:40.395039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:40.395041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:40.395311 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:40.395550 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:40.395550 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:40.395550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:40.395669 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:40.396108 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:40.396117 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:40.396315 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 22:37:40.396474 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 22:37:40.396657 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260806 22:37:40.397513 23513 heartbeater.cc:347] Connected to a master server at 127.16.235.126:46115
I20260806 22:37:40.397631 23513 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:40.397835 23513 heartbeater.cc:511] Master 127.16.235.126:46115 requested a full tablet report, sending...
I20260806 22:37:40.398252 23034 ts_manager.cc:194] Registered new tserver with Master: 7302a2eca14e4dd3ac13a59cd57880b8 (127.16.235.66:46233)
I20260806 22:37:40.398690 23034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:41217
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:40.421967 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46115
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:40.507691 23519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:40.507948 23519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:40.508019 23519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:40.509223 23519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:40.509296 23519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:40.509344 23519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:40.511020 23519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:40.511334 23519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:40.511588 23519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:40.517117 23526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:40.517165 23531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:40.517189 23529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:40.517318 23519 server_base.cc:1034] running on GCE node
I20260806 22:37:40.517633 23519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:40.517903 23519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:40.519085 23519 hybrid_clock.cc:648] HybridClock initialized: now 1786055860519068 us; error 34 us; skew 500 ppm
Aug 06 22:37:40 dist-test-slave-3kk6 krb5kdc[22965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:40.521921 23519 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:40.522383 23519 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:40.522486 23519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:40.522693 23519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:40.523916 23519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "5853db498ef14e629bceaf9c82838084"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "2dcee9ba5c49ffcc5202d4302511e489"
server_key_iv: "193e71cb35ea972dde182ec87aeffae6"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.524405 23519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "5853db498ef14e629bceaf9c82838084"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "2dcee9ba5c49ffcc5202d4302511e489"
server_key_iv: "193e71cb35ea972dde182ec87aeffae6"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.525775 23519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:40.526503 23539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.526729 23519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:40.526831 23519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "5853db498ef14e629bceaf9c82838084"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "2dcee9ba5c49ffcc5202d4302511e489"
server_key_iv: "193e71cb35ea972dde182ec87aeffae6"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.526922 23519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:40.544724 23519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:40.547341 23519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:40.547533 23519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:40.547801 23519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:40.548215 23519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:40.548272 23519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.548318 23519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:40.548345 23519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.555070 23519 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:41267
I20260806 22:37:40.555188 23692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:41267 every 8 connection(s)
I20260806 22:37:40.555501 23519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:40.558420 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 23519
I20260806 22:37:40.558557 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:40.558802 17325 external_mini_cluster.cc:1428] Setting key 07e4c3907663d5e67828fe1a0f3bcea3
Aug 06 22:37:40 dist-test-slave-3kk6 krb5kdc[22965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055860, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:40.565455 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.556386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:47115 (local address 127.16.235.126:46115)
0806 22:37:40.556449 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.556451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.556653 (+ 202us) server_negotiation.cc:407] Connection header received
0806 22:37:40.557179 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.557181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.557194 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.557267 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:40.558400 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.559131 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.560399 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.560674 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.560729 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.562997 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:40.562998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:40.562999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:40.563006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:40.564239 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.564641 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.564642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.564642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.564695 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.564927 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.564928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.564928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.565038 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:40.565091 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.565325 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.565387 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:37:40.565763 23695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.556328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:46115 (local address 127.16.235.67:47115)
0806 22:37:40.556508 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:40.556555 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:40.557053 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:40.557380 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:40.557384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:40.557675 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:40.558269 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.558277 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.559268 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.559269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.560322 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.560325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.560739 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.560740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.560855 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.561364 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:40.561372 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:40.562907 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:40.564319 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:40.564326 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:40.564328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:40.564565 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:40.564756 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:40.564757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:40.564757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:40.564870 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:40.565095 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:40.565100 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:40.565218 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 22:37:40.565313 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:37:40.565442 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260806 22:37:40.566314 23693 heartbeater.cc:347] Connected to a master server at 127.16.235.126:46115
I20260806 22:37:40.566411 23693 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:40.566540 23693 heartbeater.cc:511] Master 127.16.235.126:46115 requested a full tablet report, sending...
I20260806 22:37:40.567010 23034 ts_manager.cc:194] Registered new tserver with Master: 5853db498ef14e629bceaf9c82838084 (127.16.235.67:41267)
I20260806 22:37:40.567446 23034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:47115
I20260806 22:37:40.570797 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:37:40.578517 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.572219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59580 (local address 127.16.235.126:46115)
0806 22:37:40.572299 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.572302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.572397 (+ 95us) server_negotiation.cc:407] Connection header received
0806 22:37:40.572479 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.572481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.572491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.572550 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:40.573151 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.573666 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.574448 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.574724 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.574816 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.576052 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:40.576054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:40.576054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:40.576062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:40.577288 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.577645 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.577647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.577647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.577714 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:40.577929 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:40.577930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:40.577930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:40.578100 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:40.578216 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.578321 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.578411 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:37:40.581115 23034 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:40.583976 23034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:40.591594 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.587015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.16.235.67:41267)
0806 22:37:40.587238 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.587244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.587545 (+ 301us) server_negotiation.cc:407] Connection header received
0806 22:37:40.587597 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.587599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.587618 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.587698 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:40.588003 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.589220 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.590614 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.591158 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.591222 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.591271 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.591462 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.591507 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":88,"threads_started":1}
I20260806 22:37:40.591588 23709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.586922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:41267 (local address 127.0.0.1:53938)
0806 22:37:40.587318 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:40.587364 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:40.587446 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:40.587808 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:40.587811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:40.587813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:40.587895 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.587899 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.589484 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.589485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.590545 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.590549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.591246 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.591247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.591330 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.591346 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:37:40.591427 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:37:40.591481 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":57,"threads_started":1}
I20260806 22:37:40.592607 23707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.586962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:46233 (local address 127.0.0.1:54834)
0806 22:37:40.587406 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:40.587419 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:40.587483 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:40.588297 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:40.588298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:40.588300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:40.588360 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.588361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.589679 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.589680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.590482 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.590484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.592339 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.592340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.592416 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.592420 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:40.592495 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:37:40.592536 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":81,"threads_started":1}
I20260806 22:37:40.592633 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.587230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.16.235.66:46233)
0806 22:37:40.588104 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.588107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.588139 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:37:40.588180 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.588182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.588198 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.588284 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:40.588456 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.589566 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.591298 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.592162 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.592232 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.592305 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.592514 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.592553 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":836,"thread_start_us":93,"threads_started":1}
I20260806 22:37:40.592684 23598 tablet_service.cc:1467] Processing CreateTablet for tablet 2364925739594168babb23ec0c2afb7b (DEFAULT_TABLE table=test-table [id=e4b8f24e61904ee2a97f7a7f58485ab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:40.592959 23711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.587136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:38239 (local address 127.0.0.1:54174)
0806 22:37:40.587682 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:40.587692 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:40.587754 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:40.588003 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:40.588004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:40.588007 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:40.588059 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.588061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.590631 (+ 2570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.590632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.591477 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.591480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.592716 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.592717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.592787 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.592791 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:40.592847 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:37:40.592889 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":111,"threads_started":1}
I20260806 22:37:40.592998 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.587421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.16.235.65:38239)
0806 22:37:40.587627 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.587631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.587814 (+ 183us) server_negotiation.cc:407] Connection header received
0806 22:37:40.587857 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.587859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.587874 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.587958 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:40.588165 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.589356 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.591578 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.592545 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.592597 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.592646 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.592865 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.592916 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":93,"threads_started":1}
I20260806 22:37:40.593132 23598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2364925739594168babb23ec0c2afb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:40.593302 23410 tablet_service.cc:1467] Processing CreateTablet for tablet 2364925739594168babb23ec0c2afb7b (DEFAULT_TABLE table=test-table [id=e4b8f24e61904ee2a97f7a7f58485ab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:40.593626 23410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2364925739594168babb23ec0c2afb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:40.593597 23206 tablet_service.cc:1467] Processing CreateTablet for tablet 2364925739594168babb23ec0c2afb7b (DEFAULT_TABLE table=test-table [id=e4b8f24e61904ee2a97f7a7f58485ab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:40.593892 23206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2364925739594168babb23ec0c2afb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:40.596127 23715 tablet_bootstrap.cc:492] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: Bootstrap starting.
I20260806 22:37:40.596810 23715 tablet_bootstrap.cc:654] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:40.597205 23717 tablet_bootstrap.cc:492] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd: Bootstrap starting.
I20260806 22:37:40.597455 23718 tablet_bootstrap.cc:492] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: Bootstrap starting.
I20260806 22:37:40.597553 23715 log.cc:824] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:40.598165 23717 tablet_bootstrap.cc:654] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:40.598165 23718 tablet_bootstrap.cc:654] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:40.598426 23715 tablet_bootstrap.cc:492] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: No bootstrap required, opened a new log
I20260806 22:37:40.598512 23715 ts_tablet_manager.cc:1397] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:37:40.598815 23717 log.cc:824] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:40.598840 23718 log.cc:824] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:40.599658 23718 tablet_bootstrap.cc:492] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: No bootstrap required, opened a new log
I20260806 22:37:40.599730 23718 ts_tablet_manager.cc:1397] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:37:40.599818 23717 tablet_bootstrap.cc:492] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd: No bootstrap required, opened a new log
I20260806 22:37:40.599893 23717 ts_tablet_manager.cc:1397] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:37:40.600637 23715 raft_consensus.cc:348] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } }
I20260806 22:37:40.600842 23715 raft_consensus.cc:374] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:40.600881 23715 raft_consensus.cc:729] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5853db498ef14e629bceaf9c82838084, State: Initialized, Role: FOLLOWER
I20260806 22:37:40.601020 23715 consensus_queue.cc:260] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } }
I20260806 22:37:40.601318 23715 ts_tablet_manager.cc:1428] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:37:40.601460 23693 heartbeater.cc:503] Master 127.16.235.126:46115 was elected leader, sending a full tablet report...
I20260806 22:37:40.601828 23718 raft_consensus.cc:348] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } }
I20260806 22:37:40.601995 23718 raft_consensus.cc:374] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:40.602032 23718 raft_consensus.cc:729] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7302a2eca14e4dd3ac13a59cd57880b8, State: Initialized, Role: FOLLOWER
I20260806 22:37:40.601935 23717 raft_consensus.cc:348] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } }
I20260806 22:37:40.602104 23717 raft_consensus.cc:374] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:40.602154 23717 raft_consensus.cc:729] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243ba7d90fce4218a3ff5deb5fffa5dd, State: Initialized, Role: FOLLOWER
I20260806 22:37:40.602180 23718 consensus_queue.cc:260] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } }
I20260806 22:37:40.602294 23717 consensus_queue.cc:260] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } }
I20260806 22:37:40.602546 23718 ts_tablet_manager.cc:1428] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:37:40.602567 23513 heartbeater.cc:503] Master 127.16.235.126:46115 was elected leader, sending a full tablet report...
I20260806 22:37:40.602583 23300 heartbeater.cc:503] Master 127.16.235.126:46115 was elected leader, sending a full tablet report...
I20260806 22:37:40.602584 23717 ts_tablet_manager.cc:1428] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 22:37:40.636547 23514 tablet.cc:2369] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:40.674746 23728 raft_consensus.cc:482] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:40.674911 23728 raft_consensus.cc:504] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } }
I20260806 22:37:40.675277 23728 leader_election.cc:288] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5853db498ef14e629bceaf9c82838084 (127.16.235.67:41267), 7302a2eca14e4dd3ac13a59cd57880b8 (127.16.235.66:46233)
I20260806 22:37:40.678378 23305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.675510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:46233 (local address 127.16.235.65:56871)
0806 22:37:40.675583 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:40.675597 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:40.675682 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:40.675882 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:40.675883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:40.675886 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:40.675976 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.675978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.676638 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.676639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.677440 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.677443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.678042 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.678043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.678136 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.678139 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:40.678247 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:37:40.678296 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:37:40.678378 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.675639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:56871 (local address 127.16.235.66:46233)
0806 22:37:40.675687 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.675689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.675700 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:40.675757 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.675758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.675768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.675874 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:40.676055 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.676549 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.677546 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.677944 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.678039 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.678048 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.678262 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.678300 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:37:40.678438 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.675650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:50575 (local address 127.16.235.67:41267)
0806 22:37:40.675706 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.675709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.675905 (+ 196us) server_negotiation.cc:407] Connection header received
0806 22:37:40.675995 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.675996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.676005 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.676067 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:40.676245 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.676715 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.677644 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.678030 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.678120 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.678129 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.678332 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.678382 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:37:40.678790 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.675511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:41267 (local address 127.16.235.65:50575)
0806 22:37:40.675816 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:40.675826 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:40.675915 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:40.676084 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:40.676084 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:40.676086 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:40.676166 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.676167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.676810 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.676811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.677545 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:40.677547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.678123 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.678124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:40.678208 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.678212 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:40.678586 (+ 374us) client_negotiation.cc:240] Negotiation successful
0806 22:37:40.678640 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":74,"thread_start_us":141,"threads_started":1}
I20260806 22:37:40.678810 23435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2364925739594168babb23ec0c2afb7b" candidate_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" is_pre_election: true
I20260806 22:37:40.679021 23435 raft_consensus.cc:2454] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243ba7d90fce4218a3ff5deb5fffa5dd in term 0.
I20260806 22:37:40.679234 23624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2364925739594168babb23ec0c2afb7b" candidate_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5853db498ef14e629bceaf9c82838084" is_pre_election: true
I20260806 22:37:40.679386 23624 raft_consensus.cc:2454] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243ba7d90fce4218a3ff5deb5fffa5dd in term 0.
I20260806 22:37:40.679363 23145 leader_election.cc:302] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243ba7d90fce4218a3ff5deb5fffa5dd, 7302a2eca14e4dd3ac13a59cd57880b8; no voters:
I20260806 22:37:40.679554 23728 raft_consensus.cc:2781] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:40.679636 23728 raft_consensus.cc:482] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:40.679662 23728 raft_consensus.cc:3037] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:40.680419 23728 raft_consensus.cc:504] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } }
I20260806 22:37:40.680578 23728 leader_election.cc:288] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [CANDIDATE]: Term 1 election: Requested vote from peers 5853db498ef14e629bceaf9c82838084 (127.16.235.67:41267), 7302a2eca14e4dd3ac13a59cd57880b8 (127.16.235.66:46233)
I20260806 22:37:40.680752 23624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2364925739594168babb23ec0c2afb7b" candidate_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5853db498ef14e629bceaf9c82838084"
I20260806 22:37:40.680804 23435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2364925739594168babb23ec0c2afb7b" candidate_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7302a2eca14e4dd3ac13a59cd57880b8"
I20260806 22:37:40.680884 23435 raft_consensus.cc:3037] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:40.680892 23624 raft_consensus.cc:3037] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:40.681629 23435 raft_consensus.cc:2454] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243ba7d90fce4218a3ff5deb5fffa5dd in term 1.
I20260806 22:37:40.681654 23624 raft_consensus.cc:2454] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243ba7d90fce4218a3ff5deb5fffa5dd in term 1.
I20260806 22:37:40.681875 23150 leader_election.cc:302] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243ba7d90fce4218a3ff5deb5fffa5dd, 5853db498ef14e629bceaf9c82838084; no voters:
I20260806 22:37:40.681967 23728 raft_consensus.cc:2781] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:40.682194 23728 raft_consensus.cc:686] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [term 1 LEADER]: Becoming Leader. State: Replica: 243ba7d90fce4218a3ff5deb5fffa5dd, State: Running, Role: LEADER
I20260806 22:37:40.682320 23728 consensus_queue.cc:237] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } }
I20260806 22:37:40.683327 23030 catalog_manager.cc:5495] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 243ba7d90fce4218a3ff5deb5fffa5dd (127.16.235.65). New cstate: current_term: 1 leader_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243ba7d90fce4218a3ff5deb5fffa5dd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38239 } health_report { overall_health: HEALTHY } } }
I20260806 22:37:40.711211 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.708632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.16.235.65:38239)
0806 22:37:40.708708 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.708711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.708721 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:37:40.708774 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.708775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.708798 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.708883 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:40.709044 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.709541 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.710334 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:40.710531 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:40.710611 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:40.710838 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:40.710948 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:40.711040 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.711065 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.711089 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 22:37:40.712217 23302 tablet.cc:2369] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:40.713325 23624 raft_consensus.cc:1261] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [term 1 FOLLOWER]: Refusing update from remote peer 243ba7d90fce4218a3ff5deb5fffa5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:40.713325 23435 raft_consensus.cc:1261] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [term 1 FOLLOWER]: Refusing update from remote peer 243ba7d90fce4218a3ff5deb5fffa5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:40.713706 23731 consensus_queue.cc:1035] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5853db498ef14e629bceaf9c82838084" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:40.713860 23728 consensus_queue.cc:1035] T 2364925739594168babb23ec0c2afb7b P 243ba7d90fce4218a3ff5deb5fffa5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7302a2eca14e4dd3ac13a59cd57880b8" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:40.716418 23737 mvcc.cc:205] Tried to move back new op lower bound from 7315684805478699008 to 7315684805355569152. Current Snapshot: MvccSnapshot[applied={T|T < 7315684805478699008}]
I20260806 22:37:40.719025 23031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:40.719172 23031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:40.720953 23031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f7ff3fcafd3e4005a1eff2af806f7cfd: Sending DeleteTablet for 3 replicas of tablet 2364925739594168babb23ec0c2afb7b
I20260806 22:37:40.721776 23410 tablet_service.cc:1514] Processing DeleteTablet for tablet 2364925739594168babb23ec0c2afb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:40 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:54834
I20260806 22:37:40.722030 23206 tablet_service.cc:1514] Processing DeleteTablet for tablet 2364925739594168babb23ec0c2afb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:40 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:54174
I20260806 22:37:40.722327 23598 tablet_service.cc:1514] Processing DeleteTablet for tablet 2364925739594168babb23ec0c2afb7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:40 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:53938
I20260806 22:37:40.722586 23755 tablet_replica.cc:330] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: stopping tablet replica
I20260806 22:37:40.722788 23755 raft_consensus.cc:2229] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:40.722905 23752 tablet_replica.cc:330] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: stopping tablet replica
I20260806 22:37:40.722993 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 23123
I20260806 22:37:40.723037 23752 raft_consensus.cc:2229] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:40.723155 23752 raft_consensus.cc:2258] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:40.730561 23752 ts_tablet_manager.cc:1905] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:40.732415 23752 ts_tablet_manager.cc:1918] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:40.732532 23752 log.cc:1196] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/2364925739594168babb23ec0c2afb7b
I20260806 22:37:40.732825 23752 ts_tablet_manager.cc:1939] T 2364925739594168babb23ec0c2afb7b P 7302a2eca14e4dd3ac13a59cd57880b8: Deleting consensus metadata
I20260806 22:37:40.733350 23009 catalog_manager.cc:4841] TS 7302a2eca14e4dd3ac13a59cd57880b8 (127.16.235.66:46233): tablet 2364925739594168babb23ec0c2afb7b (table test-table [id=e4b8f24e61904ee2a97f7a7f58485ab5]) successfully deleted
I20260806 22:37:40.734925 23755 raft_consensus.cc:2258] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 22:37:40.735275 23014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:38239 (error 108)
I20260806 22:37:40.735618 23755 ts_tablet_manager.cc:1905] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:40.735692 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 23310
I20260806 22:37:40.738166 23711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.735767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:38239 (local address 127.0.0.1:54190)
0806 22:37:40.738058 (+ 2291us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:40.738118 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:38239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2264}
I20260806 22:37:40.738174 23755 ts_tablet_manager.cc:1918] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:40.738270 23755 log.cc:1196] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/2364925739594168babb23ec0c2afb7b
W20260806 22:37:40.738468 23014 catalog_manager.cc:4584] TS 243ba7d90fce4218a3ff5deb5fffa5dd (127.16.235.65:38239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2364925739594168babb23ec0c2afb7b: Network error: Client connection negotiation failed: client connection to 127.16.235.65:38239: connect: Connection refused (error 111)
I20260806 22:37:40.738565 23755 ts_tablet_manager.cc:1939] T 2364925739594168babb23ec0c2afb7b P 5853db498ef14e629bceaf9c82838084: Deleting consensus metadata
I20260806 22:37:40.739174 23014 catalog_manager.cc:4841] TS 5853db498ef14e629bceaf9c82838084 (127.16.235.67:41267): tablet 2364925739594168babb23ec0c2afb7b (table test-table [id=e4b8f24e61904ee2a97f7a7f58485ab5]) successfully deleted
I20260806 22:37:40.747287 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 23519
I20260806 22:37:40.760259 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 22984
2026-08-06T22:37:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2238 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T22:37:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:40Z Disabled control of system clock
I20260806 22:37:40.796326 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40773
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:40773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:37:40.882296 23761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:40.882588 23761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:40.882628 23761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:40.883862 23761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:40.883942 23761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:40.883968 23761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:40.884002 23761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:40.884027 23761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:40.885428 23761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:40773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:40.885771 23761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:40.886112 23761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:40.891160 23774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:40.891181 23769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:40.891242 23768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:40.891477 23761 server_base.cc:1034] running on GCE node
I20260806 22:37:40.891835 23761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:40.892159 23761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:40.893345 23761 hybrid_clock.cc:648] HybridClock initialized: now 1786055860893331 us; error 34 us; skew 500 ppm
I20260806 22:37:40.898557 23761 webserver.cc:460] Webserver started at http://127.16.235.126:39675/ using document root <none> and password file <none>
I20260806 22:37:40.898911 23761 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:40.898973 23761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:40.899190 23761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:40.900411 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "2d5e8e1ad4ee43db99e214fbf1d59230"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "9a88dc1f0b0fd645a46b737b1f8bfbf8"
server_key_iv: "1bbbc8fd09b7f30c916ac3061ee65b86"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.900817 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "2d5e8e1ad4ee43db99e214fbf1d59230"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "9a88dc1f0b0fd645a46b737b1f8bfbf8"
server_key_iv: "1bbbc8fd09b7f30c916ac3061ee65b86"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.902330 23761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:40.903218 23784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:40.903414 23761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:37:40.903513 23761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "2d5e8e1ad4ee43db99e214fbf1d59230"
format_stamp: "Formatted at 2026-08-06 22:37:40 on dist-test-slave-3kk6"
server_key: "9a88dc1f0b0fd645a46b737b1f8bfbf8"
server_key_iv: "1bbbc8fd09b7f30c916ac3061ee65b86"
server_key_version: "encryptionkey@0"
I20260806 22:37:40.903596 23761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:40.907177 23761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:40.907473 23761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:40.907629 23761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:40.911965 23761 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:40773
I20260806 22:37:40.912031 23859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:40773 every 8 connection(s)
I20260806 22:37:40.912400 23761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:40.913103 23860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:40.916270 23860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230: Bootstrap starting.
I20260806 22:37:40.916905 23860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:40.917244 23860 log.cc:824] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:40.918023 23860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230: No bootstrap required, opened a new log
I20260806 22:37:40.919879 23860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e8e1ad4ee43db99e214fbf1d59230" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40773 } }
I20260806 22:37:40.920033 23860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:40.920056 23860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5e8e1ad4ee43db99e214fbf1d59230, State: Initialized, Role: FOLLOWER
I20260806 22:37:40.920323 23860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e8e1ad4ee43db99e214fbf1d59230" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40773 } }
I20260806 22:37:40.920431 23860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:40.920480 23860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:40.920532 23860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:40.921202 23860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e8e1ad4ee43db99e214fbf1d59230" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40773 } }
I20260806 22:37:40.921331 23860 leader_election.cc:302] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d5e8e1ad4ee43db99e214fbf1d59230; no voters:
I20260806 22:37:40.921545 23860 leader_election.cc:288] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:40.921597 23864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:40.921799 23864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [term 1 LEADER]: Becoming Leader. State: Replica: 2d5e8e1ad4ee43db99e214fbf1d59230, State: Running, Role: LEADER
I20260806 22:37:40.921883 23860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:40.921898 23864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e8e1ad4ee43db99e214fbf1d59230" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40773 } }
I20260806 22:37:40.922050 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 23761
I20260806 22:37:40.922147 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:40.922400 17325 external_mini_cluster.cc:1428] Setting key b0a2f6352125fc6f8e41595135a1d1d2
I20260806 22:37:40.923391 23866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d5e8e1ad4ee43db99e214fbf1d59230. Latest consensus state: current_term: 1 leader_uuid: "2d5e8e1ad4ee43db99e214fbf1d59230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e8e1ad4ee43db99e214fbf1d59230" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40773 } } }
I20260806 22:37:40.923498 23866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:40.923564 23865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d5e8e1ad4ee43db99e214fbf1d59230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e8e1ad4ee43db99e214fbf1d59230" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40773 } } }
I20260806 22:37:40.923663 23865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:40.923753 23881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:40.924299 23881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:40.925737 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:40.923186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.16.235.126:40773)
0806 22:37:40.923393 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:40.923403 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:40.923978 (+ 575us) server_negotiation.cc:407] Connection header received
0806 22:37:40.924234 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:40.924243 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:40.924293 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:40.924452 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:40.925202 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:40.925205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:40.925206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:40.925211 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:40.925256 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:40.925271 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:40.925353 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:40.925482 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 22:37:40.925555 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":102,"threads_started":1}
I20260806 22:37:40.926396 23881 catalog_manager.cc:1347] Generated new cluster ID: 8cedb0846db04f989d658cd8de09ecc0
I20260806 22:37:40.926455 23881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:40.945379 23881 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:40.945647 23881 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:40.949355 23881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230: Generated new TSK 0
I20260806 22:37:40.949595 23881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:37:40.956739 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40773
--builtin_ntp_servers=127.16.235.84:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:37:41.044229 23891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:41.044476 23891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:41.044502 23891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:41.045735 23891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:41.045784 23891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:41.045816 23891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:41.047341 23891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:41.047601 23891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:41.047946 23891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:37:41.053719 23891 server_base.cc:1034] running on GCE node
W20260806 22:37:41.053669 23903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.053690 23901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.053679 23900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:41.054176 23891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:41.054407 23891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:41.055579 23891 hybrid_clock.cc:648] HybridClock initialized: now 1786055861055567 us; error 46 us; skew 500 ppm
I20260806 22:37:41.057046 23891 webserver.cc:460] Webserver started at http://127.16.235.65:42715/ using document root <none> and password file <none>
I20260806 22:37:41.057307 23891 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:41.057384 23891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:41.057545 23891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:41.058779 23891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "dc1d243abeda4402831cc34a4d33acf8"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "c2581740fcb23a991cb6764cce899afa"
server_key_iv: "4fa7e05a162052c5b8edcdc1544c31e4"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.059178 23891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "dc1d243abeda4402831cc34a4d33acf8"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "c2581740fcb23a991cb6764cce899afa"
server_key_iv: "4fa7e05a162052c5b8edcdc1544c31e4"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.060577 23891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:41.061369 23911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.061543 23891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:41.061656 23891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "dc1d243abeda4402831cc34a4d33acf8"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "c2581740fcb23a991cb6764cce899afa"
server_key_iv: "4fa7e05a162052c5b8edcdc1544c31e4"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.061735 23891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:41.065307 23891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:41.065585 23891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:41.065733 23891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:41.065960 23891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:41.066287 23891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:41.066342 23891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.066385 23891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:41.066416 23891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.072798 23891 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:37761
I20260806 22:37:41.072839 24061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:37761 every 8 connection(s)
I20260806 22:37:41.073207 23891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:41.076566 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.073984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:51077 (local address 127.16.235.126:40773)
0806 22:37:41.074031 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.074034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.074254 (+ 220us) server_negotiation.cc:407] Connection header received
0806 22:37:41.074882 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.074884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.074894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.074953 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.076298 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.076300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.076300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.076305 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.076324 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.076330 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.076394 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.076474 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.076494 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:37:41.076771 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.073947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40773 (local address 127.16.235.65:51077)
0806 22:37:41.074128 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.074163 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.074753 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.075029 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.075033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.075043 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:41.076134 (+ 1091us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:37:41.076134 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:41.076163 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:37:41.076193 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:41.076396 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:41.076403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:41.076405 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:37:41.076476 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:37:41.076541 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260806 22:37:41.077281 24062 heartbeater.cc:347] Connected to a master server at 127.16.235.126:40773
I20260806 22:37:41.077368 24062 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:41.077486 24062 heartbeater.cc:511] Master 127.16.235.126:40773 requested a full tablet report, sending...
I20260806 22:37:41.077963 23808 ts_manager.cc:194] Registered new tserver with Master: dc1d243abeda4402831cc34a4d33acf8 (127.16.235.65:37761)
I20260806 22:37:41.082947 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 23891
I20260806 22:37:41.083048 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:41.083266 17325 external_mini_cluster.cc:1428] Setting key e8723d6ad69810b3369c5c66e4a3b0d0
I20260806 22:37:41.084353 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40773
--builtin_ntp_servers=127.16.235.84:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:37:41.172238 24065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:41.172473 24065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:41.172498 24065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:41.173637 24065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:41.173677 24065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:41.173703 24065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:41.175125 24065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:41.175346 24065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:41.175628 24065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:41.181034 24070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.181047 24071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.181047 24073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:41.181352 24065 server_base.cc:1034] running on GCE node
I20260806 22:37:41.181638 24065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:41.181946 24065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:41.183122 24065 hybrid_clock.cc:648] HybridClock initialized: now 1786055861183101 us; error 60 us; skew 500 ppm
I20260806 22:37:41.184720 24065 webserver.cc:460] Webserver started at http://127.16.235.66:40745/ using document root <none> and password file <none>
I20260806 22:37:41.185011 24065 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:41.185091 24065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:41.185258 24065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:41.186440 24065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "4f629cd1b8bd4aa9bb690e39246c7e74"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "ecdfd9ca85f6a74fd8573a3cdd6c8d10"
server_key_iv: "6378556a10666019c78ec2a2cde791b6"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.186888 24065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "4f629cd1b8bd4aa9bb690e39246c7e74"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "ecdfd9ca85f6a74fd8573a3cdd6c8d10"
server_key_iv: "6378556a10666019c78ec2a2cde791b6"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.188334 24065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:41.189218 24085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.189426 24065 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:41.189543 24065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "4f629cd1b8bd4aa9bb690e39246c7e74"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "ecdfd9ca85f6a74fd8573a3cdd6c8d10"
server_key_iv: "6378556a10666019c78ec2a2cde791b6"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.189625 24065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:41.193394 24065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:41.193682 24065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:41.193840 24065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:41.194052 24065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:41.194396 24065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:41.194453 24065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.194494 24065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:41.194521 24065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.200956 24065 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:45229
I20260806 22:37:41.201004 24245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:45229 every 8 connection(s)
I20260806 22:37:41.201395 24065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:41.205084 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.202287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:60373 (local address 127.16.235.126:40773)
0806 22:37:41.202416 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.202419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.202708 (+ 289us) server_negotiation.cc:407] Connection header received
0806 22:37:41.203310 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.203312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.203319 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.203396 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.204838 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.204840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.204840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.204846 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.204865 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.204869 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.204935 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.205012 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.205026 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:37:41.205279 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.202206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40773 (local address 127.16.235.66:60373)
0806 22:37:41.202547 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.202593 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.203169 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.203530 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.203534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.203542 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:41.204654 (+ 1112us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:37:41.204654 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:41.204686 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:37:41.204718 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:41.204945 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:41.204948 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:41.204950 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:37:41.205008 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:37:41.205062 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":141,"threads_started":1}
I20260806 22:37:41.205842 24246 heartbeater.cc:347] Connected to a master server at 127.16.235.126:40773
I20260806 22:37:41.205982 24246 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:41.206151 24246 heartbeater.cc:511] Master 127.16.235.126:40773 requested a full tablet report, sending...
I20260806 22:37:41.206579 23808 ts_manager.cc:194] Registered new tserver with Master: 4f629cd1b8bd4aa9bb690e39246c7e74 (127.16.235.66:45229)
I20260806 22:37:41.210963 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 24065
I20260806 22:37:41.211107 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:41.211301 17325 external_mini_cluster.cc:1428] Setting key c6f5f3e0afdc8d65f27d1016f746a73a
I20260806 22:37:41.212402 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40773
--builtin_ntp_servers=127.16.235.84:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:37:41.300969 24250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:41.301193 24250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:41.301218 24250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:41.302389 24250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:41.302436 24250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:41.302464 24250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:41.304013 24250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:40773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:41.304277 24250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:41.304643 24250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:41.309916 24256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.309918 24257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.309983 24262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:41.310247 24250 server_base.cc:1034] running on GCE node
I20260806 22:37:41.310464 24250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:41.310729 24250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:41.311901 24250 hybrid_clock.cc:648] HybridClock initialized: now 1786055861311888 us; error 31 us; skew 500 ppm
I20260806 22:37:41.313337 24250 webserver.cc:460] Webserver started at http://127.16.235.67:46639/ using document root <none> and password file <none>
I20260806 22:37:41.313596 24250 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:41.313647 24250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:41.313813 24250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:41.315032 24250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "437dea60577f4e649209a7a0389c8312"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "39ee90758e8171fd0f5246127ed73e4b"
server_key_iv: "36a1913c857929c46c3e01293535cf9e"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.315418 24250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "437dea60577f4e649209a7a0389c8312"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "39ee90758e8171fd0f5246127ed73e4b"
server_key_iv: "36a1913c857929c46c3e01293535cf9e"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.316897 24250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:41.317732 24271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.317950 24250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:41.318028 24250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "437dea60577f4e649209a7a0389c8312"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "39ee90758e8171fd0f5246127ed73e4b"
server_key_iv: "36a1913c857929c46c3e01293535cf9e"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.318123 24250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:41.321762 24250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:41.322067 24250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:41.322234 24250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:41.322486 24250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:41.322916 24250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:41.322956 24250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.322978 24250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:41.322991 24250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.329248 24250 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:41285
I20260806 22:37:41.329290 24434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:41285 every 8 connection(s)
I20260806 22:37:41.329649 24250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:41.332973 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.330594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:42467 (local address 127.16.235.126:40773)
0806 22:37:41.330701 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.330703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.330939 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:37:41.331419 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.331420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.331426 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.331498 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.332785 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.332787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.332787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.332791 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.332804 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.332808 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.332857 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.332903 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.332916 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:37:41.333212 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.330577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40773 (local address 127.16.235.67:42467)
0806 22:37:41.330798 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.330842 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.331325 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.331560 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.331563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.331571 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:41.332633 (+ 1062us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:37:41.332633 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:41.332662 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:37:41.332694 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:41.332855 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:41.332857 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:41.332860 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:41.332909 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:37:41.332958 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":125,"threads_started":1}
I20260806 22:37:41.333740 24436 heartbeater.cc:347] Connected to a master server at 127.16.235.126:40773
I20260806 22:37:41.333841 24436 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:41.334010 24436 heartbeater.cc:511] Master 127.16.235.126:40773 requested a full tablet report, sending...
I20260806 22:37:41.334489 23808 ts_manager.cc:194] Registered new tserver with Master: 437dea60577f4e649209a7a0389c8312 (127.16.235.67:41285)
I20260806 22:37:41.338912 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 24250
I20260806 22:37:41.339002 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:41.339151 17325 external_mini_cluster.cc:1428] Setting key 13c4ba5fa4ab5bd725786c3854fd1461
I20260806 22:37:41.340234 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:37:41.343124 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.341671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.16.235.126:40773)
0806 22:37:41.341810 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.341812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.341929 (+ 117us) server_negotiation.cc:407] Connection header received
0806 22:37:41.342017 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.342018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.342027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.342100 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.342860 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.342861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.342862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.342866 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.342885 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.342889 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.342964 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.343027 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.343045 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 22:37:41.345031 23808 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:41.347025 23808 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:41.351302 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.349579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.16.235.67:41285)
0806 22:37:41.349956 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.349962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.349973 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:41.350001 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.350003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.350013 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.350388 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.350953 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.350960 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.350961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.350966 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.351001 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.351027 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.351107 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.351200 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.351216 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":90,"threads_started":1}
I20260806 22:37:41.351285 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.349438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:41285 (local address 127.0.0.1:47212)
0806 22:37:41.349645 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.349688 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.349747 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.350138 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.350145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.350151 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:41.350831 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:37:41.350832 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:41.350846 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:37:41.350866 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:41.351115 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:41.351122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:41.351125 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:41.351194 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:37:41.351210 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":91,"threads_started":1}
I20260806 22:37:41.351361 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.349563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.16.235.66:45229)
0806 22:37:41.350015 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.350020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.350031 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:41.350061 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.350063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.350073 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.350394 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.351030 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.351039 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.351039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.351043 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.351078 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.351100 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.351187 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.351257 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.351271 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":109,"threads_started":1}
I20260806 22:37:41.351380 24446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.349439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:45229 (local address 127.0.0.1:42690)
0806 22:37:41.349768 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.349781 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.349839 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.350196 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.350197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.350199 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:41.350915 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:37:41.350916 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:41.350923 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:37:41.350938 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:41.351185 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:41.351186 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:41.351187 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:37:41.351247 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:37:41.351262 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":114,"threads_started":1}
I20260806 22:37:41.352272 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.349673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:37761 (local address 127.0.0.1:52334)
0806 22:37:41.349786 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.349796 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.349839 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.350568 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.350569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.350572 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:41.351858 (+ 1286us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:37:41.351859 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:41.351867 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:37:41.351881 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:41.352128 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:41.352129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:41.352130 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:37:41.352181 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:37:41.352196 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260806 22:37:41.352314 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.349892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.16.235.65:37761)
0806 22:37:41.350105 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.350108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.350117 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:41.350142 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.350144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.350152 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.350224 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.351969 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.351981 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.351982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.351987 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.352021 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.352048 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.352124 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.352200 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.352217 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":100,"threads_started":1}
I20260806 22:37:41.352279 24341 tablet_service.cc:1467] Processing CreateTablet for tablet 7bff87a2be09421faaa73ebbe4120633 (DEFAULT_TABLE table=test-table [id=14d80d7c114b410f95b53e72b76f08c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:41.352398 24157 tablet_service.cc:1467] Processing CreateTablet for tablet 7bff87a2be09421faaa73ebbe4120633 (DEFAULT_TABLE table=test-table [id=14d80d7c114b410f95b53e72b76f08c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:41.352702 24341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bff87a2be09421faaa73ebbe4120633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:41.352725 24157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bff87a2be09421faaa73ebbe4120633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:41.355388 24459 tablet_bootstrap.cc:492] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: Bootstrap starting.
I20260806 22:37:41.355399 24460 tablet_bootstrap.cc:492] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: Bootstrap starting.
I20260806 22:37:41.356096 24459 tablet_bootstrap.cc:654] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:41.356129 24460 tablet_bootstrap.cc:654] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:41.356273 23978 tablet_service.cc:1467] Processing CreateTablet for tablet 7bff87a2be09421faaa73ebbe4120633 (DEFAULT_TABLE table=test-table [id=14d80d7c114b410f95b53e72b76f08c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:41.356426 24459 log.cc:824] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:41.356458 24460 log.cc:824] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:41.356674 23978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bff87a2be09421faaa73ebbe4120633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:41.357307 24460 tablet_bootstrap.cc:492] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: No bootstrap required, opened a new log
I20260806 22:37:41.357386 24460 ts_tablet_manager.cc:1397] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:41.357551 24459 tablet_bootstrap.cc:492] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: No bootstrap required, opened a new log
I20260806 22:37:41.357637 24459 ts_tablet_manager.cc:1397] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:37:41.359544 24459 raft_consensus.cc:348] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } }
I20260806 22:37:41.359706 24459 raft_consensus.cc:374] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:41.359754 24459 raft_consensus.cc:729] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f629cd1b8bd4aa9bb690e39246c7e74, State: Initialized, Role: FOLLOWER
I20260806 22:37:41.359897 24459 consensus_queue.cc:260] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } }
I20260806 22:37:41.360064 24460 raft_consensus.cc:348] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } }
I20260806 22:37:41.360198 24459 ts_tablet_manager.cc:1428] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:37:41.360244 24460 raft_consensus.cc:374] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:41.360288 24460 raft_consensus.cc:729] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437dea60577f4e649209a7a0389c8312, State: Initialized, Role: FOLLOWER
I20260806 22:37:41.360339 24246 heartbeater.cc:503] Master 127.16.235.126:40773 was elected leader, sending a full tablet report...
I20260806 22:37:41.360394 24460 consensus_queue.cc:260] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } }
I20260806 22:37:41.360559 24466 tablet_bootstrap.cc:492] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8: Bootstrap starting.
I20260806 22:37:41.360683 24460 ts_tablet_manager.cc:1428] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:37:41.360809 24436 heartbeater.cc:503] Master 127.16.235.126:40773 was elected leader, sending a full tablet report...
I20260806 22:37:41.361340 24466 tablet_bootstrap.cc:654] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:41.361626 24466 log.cc:824] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:41.362473 24466 tablet_bootstrap.cc:492] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8: No bootstrap required, opened a new log
I20260806 22:37:41.362553 24466 ts_tablet_manager.cc:1397] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:41.364696 24466 raft_consensus.cc:348] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } }
I20260806 22:37:41.364846 24466 raft_consensus.cc:374] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:41.364895 24466 raft_consensus.cc:729] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc1d243abeda4402831cc34a4d33acf8, State: Initialized, Role: FOLLOWER
I20260806 22:37:41.365015 24466 consensus_queue.cc:260] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } }
I20260806 22:37:41.365262 24466 ts_tablet_manager.cc:1428] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:37:41.365331 24062 heartbeater.cc:503] Master 127.16.235.126:40773 was elected leader, sending a full tablet report...
W20260806 22:37:41.452144 24247 tablet.cc:2369] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:41.499156 24467 raft_consensus.cc:482] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:41.499352 24467 raft_consensus.cc:504] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } }
I20260806 22:37:41.499748 24467 leader_election.cc:288] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f629cd1b8bd4aa9bb690e39246c7e74 (127.16.235.66:45229), dc1d243abeda4402831cc34a4d33acf8 (127.16.235.65:37761)
I20260806 22:37:41.501297 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.499998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:45229 (local address 127.16.235.67:37417)
0806 22:37:41.500062 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.500076 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.500120 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.500319 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.500321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.500324 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:41.500853 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:37:41.500853 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:41.500862 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:37:41.500891 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:41.501134 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:41.501135 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:41.501138 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:41.501198 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:37:41.501213 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:37:41.501334 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.500078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:37417 (local address 127.16.235.66:45229)
0806 22:37:41.500148 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.500152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.500165 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:37:41.500199 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.500200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.500207 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.500295 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.501010 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.501012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.501012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.501018 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.501041 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.501046 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.501115 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.501219 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.501232 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:37:41.501448 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.500140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:37761 (local address 127.16.235.67:50405)
0806 22:37:41.500373 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.500383 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.500454 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.500601 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.500602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.500605 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:41.501058 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:37:41.501059 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:41.501066 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:37:41.501080 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:41.501286 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:41.501287 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:41.501288 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:37:41.501343 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:37:41.501357 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":153,"threads_started":1}
I20260806 22:37:41.501451 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.500274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:50405 (local address 127.16.235.65:37761)
0806 22:37:41.500326 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.500328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.500451 (+ 123us) server_negotiation.cc:407] Connection header received
0806 22:37:41.500514 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.500515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.500522 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.500596 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.501176 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.501178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.501179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.501184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.501204 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.501210 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.501266 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.501342 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.501355 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:37:41.501653 24186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bff87a2be09421faaa73ebbe4120633" candidate_uuid: "437dea60577f4e649209a7a0389c8312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" is_pre_election: true
I20260806 22:37:41.501824 24186 raft_consensus.cc:2454] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437dea60577f4e649209a7a0389c8312 in term 0.
I20260806 22:37:41.501953 24003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bff87a2be09421faaa73ebbe4120633" candidate_uuid: "437dea60577f4e649209a7a0389c8312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc1d243abeda4402831cc34a4d33acf8" is_pre_election: true
I20260806 22:37:41.502095 24277 leader_election.cc:302] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437dea60577f4e649209a7a0389c8312, 4f629cd1b8bd4aa9bb690e39246c7e74; no voters:
I20260806 22:37:41.502113 24003 raft_consensus.cc:2454] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 437dea60577f4e649209a7a0389c8312 in term 0.
I20260806 22:37:41.502219 24467 raft_consensus.cc:2781] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:41.502285 24467 raft_consensus.cc:482] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:41.502324 24467 raft_consensus.cc:3037] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:41.503288 24467 raft_consensus.cc:504] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } }
I20260806 22:37:41.503430 24467 leader_election.cc:288] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [CANDIDATE]: Term 1 election: Requested vote from peers 4f629cd1b8bd4aa9bb690e39246c7e74 (127.16.235.66:45229), dc1d243abeda4402831cc34a4d33acf8 (127.16.235.65:37761)
I20260806 22:37:41.503595 24186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bff87a2be09421faaa73ebbe4120633" candidate_uuid: "437dea60577f4e649209a7a0389c8312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74"
I20260806 22:37:41.503615 24003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bff87a2be09421faaa73ebbe4120633" candidate_uuid: "437dea60577f4e649209a7a0389c8312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc1d243abeda4402831cc34a4d33acf8"
I20260806 22:37:41.503690 24003 raft_consensus.cc:3037] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:41.503690 24186 raft_consensus.cc:3037] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:41.504613 24003 raft_consensus.cc:2454] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437dea60577f4e649209a7a0389c8312 in term 1.
I20260806 22:37:41.504613 24186 raft_consensus.cc:2454] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 437dea60577f4e649209a7a0389c8312 in term 1.
I20260806 22:37:41.504784 24277 leader_election.cc:302] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437dea60577f4e649209a7a0389c8312, 4f629cd1b8bd4aa9bb690e39246c7e74; no voters:
I20260806 22:37:41.504917 24467 raft_consensus.cc:2781] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:41.505086 24467 raft_consensus.cc:686] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 1 LEADER]: Becoming Leader. State: Replica: 437dea60577f4e649209a7a0389c8312, State: Running, Role: LEADER
I20260806 22:37:41.505192 24467 consensus_queue.cc:237] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } }
I20260806 22:37:41.506122 23808 catalog_manager.cc:5495] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 reported cstate change: term changed from 0 to 1, leader changed from <none> to 437dea60577f4e649209a7a0389c8312 (127.16.235.67). New cstate: current_term: 1 leader_uuid: "437dea60577f4e649209a7a0389c8312" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "437dea60577f4e649209a7a0389c8312" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 41285 } health_report { overall_health: HEALTHY } } }
I20260806 22:37:41.537591 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.536445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.16.235.67:41285)
0806 22:37:41.536526 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.536527 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.536535 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:41.536575 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.536575 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.536590 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.536659 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.537356 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.537357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.537358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.537362 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.537378 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.537382 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.537442 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.537492 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.537508 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:37:41.539697 24003 raft_consensus.cc:1261] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [term 1 FOLLOWER]: Refusing update from remote peer 437dea60577f4e649209a7a0389c8312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:41.539705 24186 raft_consensus.cc:1261] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [term 1 FOLLOWER]: Refusing update from remote peer 437dea60577f4e649209a7a0389c8312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:41.540078 24467 consensus_queue.cc:1035] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f629cd1b8bd4aa9bb690e39246c7e74" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:41.540253 24471 consensus_queue.cc:1035] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc1d243abeda4402831cc34a4d33acf8" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 37761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:41.543216 24475 mvcc.cc:205] Tried to move back new op lower bound from 7315684808863408128 to 7315684808726093824. Current Snapshot: MvccSnapshot[applied={T|T < 7315684808863408128}]
I20260806 22:37:41.544052 24477 mvcc.cc:205] Tried to move back new op lower bound from 7315684808863408128 to 7315684808726093824. Current Snapshot: MvccSnapshot[applied={T|T < 7315684808863408128}]
I20260806 22:37:41.545497 23808 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:41.545607 23808 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:41.546399 23808 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d5e8e1ad4ee43db99e214fbf1d59230: Sending DeleteTablet for 3 replicas of tablet 7bff87a2be09421faaa73ebbe4120633
I20260806 22:37:41.546772 24157 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bff87a2be09421faaa73ebbe4120633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:41 UTC) from {username='slave'} at 127.0.0.1:42690
I20260806 22:37:41.546828 23978 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bff87a2be09421faaa73ebbe4120633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:41 UTC) from {username='slave'} at 127.0.0.1:52334
I20260806 22:37:41.546808 24341 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bff87a2be09421faaa73ebbe4120633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:41 UTC) from {username='slave'} at 127.0.0.1:47212
I20260806 22:37:41.547039 24486 tablet_replica.cc:330] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: stopping tablet replica
I20260806 22:37:41.547148 24489 tablet_replica.cc:330] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8: stopping tablet replica
I20260806 22:37:41.547153 24486 raft_consensus.cc:2229] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:41.547286 24489 raft_consensus.cc:2229] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:41.547298 24486 raft_consensus.cc:2258] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:41.547298 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 23891
I20260806 22:37:41.547406 24489 raft_consensus.cc:2258] T 7bff87a2be09421faaa73ebbe4120633 P dc1d243abeda4402831cc34a4d33acf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:41.547559 24487 tablet_replica.cc:330] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: stopping tablet replica
I20260806 22:37:41.547644 24487 raft_consensus.cc:2229] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:41.547727 24487 raft_consensus.cc:2258] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:41.550422 24486 ts_tablet_manager.cc:1905] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:41.553015 24486 ts_tablet_manager.cc:1918] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:41.553098 24486 log.cc:1196] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/7bff87a2be09421faaa73ebbe4120633
I20260806 22:37:41.553342 24486 ts_tablet_manager.cc:1939] T 7bff87a2be09421faaa73ebbe4120633 P 4f629cd1b8bd4aa9bb690e39246c7e74: Deleting consensus metadata
I20260806 22:37:41.553821 23790 catalog_manager.cc:4841] TS 4f629cd1b8bd4aa9bb690e39246c7e74 (127.16.235.66:45229): tablet 7bff87a2be09421faaa73ebbe4120633 (table test-table [id=14d80d7c114b410f95b53e72b76f08c1]) successfully deleted
I20260806 22:37:41.555015 24487 ts_tablet_manager.cc:1905] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:41.557101 24487 ts_tablet_manager.cc:1918] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:41.557190 24487 log.cc:1196] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/7bff87a2be09421faaa73ebbe4120633
I20260806 22:37:41.557500 24487 ts_tablet_manager.cc:1939] T 7bff87a2be09421faaa73ebbe4120633 P 437dea60577f4e649209a7a0389c8312: Deleting consensus metadata
I20260806 22:37:41.558051 23785 catalog_manager.cc:4841] TS 437dea60577f4e649209a7a0389c8312 (127.16.235.67:41285): tablet 7bff87a2be09421faaa73ebbe4120633 (table test-table [id=14d80d7c114b410f95b53e72b76f08c1]) successfully deleted
W20260806 22:37:41.560091 23790 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:37761 (error 108)
I20260806 22:37:41.560263 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 24065
I20260806 22:37:41.567394 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.567168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:37761 (local address 127.0.0.1:52342)
0806 22:37:41.567245 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.567332 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:37761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 22:37:41.567610 23790 catalog_manager.cc:4584] TS dc1d243abeda4402831cc34a4d33acf8 (127.16.235.65:37761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7bff87a2be09421faaa73ebbe4120633: Network error: Client connection negotiation failed: client connection to 127.16.235.65:37761: connect: Connection refused (error 111)
I20260806 22:37:41.580725 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 24250
I20260806 22:37:41.591619 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 23761
2026-08-06T22:37:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (831 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 22:37:41.618173 17325 mini_oidc.cc:152] Starting JWKS server
I20260806 22:37:41.619544 17325 webserver.cc:460] Webserver started at https://127.0.0.1:40177/ using document root <none> and password file <none>
I20260806 22:37:41.619637 17325 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:37:41.619845 17325 webserver.cc:460] Webserver started at http://127.0.0.1:37803/ using document root <none> and password file <none>
2026-08-06T22:37:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:41Z Disabled control of system clock
I20260806 22:37:41.629845 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33419
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:36741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:33419
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40177/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/testchainca.cert with env {}
W20260806 22:37:41.716572 24496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:41.716822 24496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:41.716846 24496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:41.718089 24496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:41.718139 24496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:41.718152 24496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:41.718165 24496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:41.718221 24496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:37:41.718240 24496 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:37:41.718261 24496 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40177/jwks.json
I20260806 22:37:41.719990 24496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:33419
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40177/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:41.720278 24496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:41.720538 24496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:41.726508 24502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.726557 24501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.726871 24504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:41.727186 24496 server_base.cc:1034] running on GCE node
I20260806 22:37:41.727483 24496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:41.727772 24496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:41.728971 24496 hybrid_clock.cc:648] HybridClock initialized: now 1786055861728961 us; error 41 us; skew 500 ppm
I20260806 22:37:41.730461 24496 webserver.cc:460] Webserver started at http://127.16.235.126:44759/ using document root <none> and password file <none>
I20260806 22:37:41.730824 24496 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:41.730906 24496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:41.731078 24496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:41.732362 24496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "15dd0c307c8840f59ccea5450ebc07e1"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "2469c1c74aeba22e63070779a6ce2392"
server_key_iv: "120f2528964ab38f663af14ad3a53e3a"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.732818 24496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "15dd0c307c8840f59ccea5450ebc07e1"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "2469c1c74aeba22e63070779a6ce2392"
server_key_iv: "120f2528964ab38f663af14ad3a53e3a"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.734344 24496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:41.735234 24512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.735420 24496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:37:41.735539 24496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "15dd0c307c8840f59ccea5450ebc07e1"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "2469c1c74aeba22e63070779a6ce2392"
server_key_iv: "120f2528964ab38f663af14ad3a53e3a"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.735623 24496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:41.758245 24496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:41.758702 24496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:41.758893 24496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:41.763377 24496 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:33419
I20260806 22:37:41.763434 24581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:33419 every 8 connection(s)
I20260806 22:37:41.763851 24496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:41.764990 24583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:41.765436 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 24496
I20260806 22:37:41.765571 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:41.765790 17325 external_mini_cluster.cc:1428] Setting key 0e43ebed60c18804492d2d538ce409b8
I20260806 22:37:41.768139 24583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1: Bootstrap starting.
I20260806 22:37:41.768860 24583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:41.769186 24583 log.cc:824] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:41.770040 24583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1: No bootstrap required, opened a new log
I20260806 22:37:41.771874 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.766424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.16.235.126:33419)
0806 22:37:41.766628 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.766636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.766714 (+ 78us) server_negotiation.cc:407] Connection header received
0806 22:37:41.767154 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.767160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.767223 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.767351 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:41.767615 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.768587 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.769696 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.770152 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.770323 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:41.771162 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:41.771166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:41.771167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:41.771367 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:41.771403 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:41.771414 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:41.771498 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:41.771586 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:37:41.771685 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260806 22:37:41.772347 24583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15dd0c307c8840f59ccea5450ebc07e1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33419 } }
I20260806 22:37:41.772552 24583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:41.772598 24583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15dd0c307c8840f59ccea5450ebc07e1, State: Initialized, Role: FOLLOWER
I20260806 22:37:41.772717 24583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15dd0c307c8840f59ccea5450ebc07e1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33419 } }
I20260806 22:37:41.772822 24583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:41.772867 24583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:41.772892 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:37:41.772914 24583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:41.773563 24583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15dd0c307c8840f59ccea5450ebc07e1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33419 } }
I20260806 22:37:41.773692 24583 leader_election.cc:302] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15dd0c307c8840f59ccea5450ebc07e1; no voters:
I20260806 22:37:41.773897 24583 leader_election.cc:288] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:41.773932 24589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:41.774091 24589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [term 1 LEADER]: Becoming Leader. State: Replica: 15dd0c307c8840f59ccea5450ebc07e1, State: Running, Role: LEADER
I20260806 22:37:41.774183 24589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15dd0c307c8840f59ccea5450ebc07e1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33419 } }
I20260806 22:37:41.774250 24583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:41.774552 24590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15dd0c307c8840f59ccea5450ebc07e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15dd0c307c8840f59ccea5450ebc07e1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33419 } } }
I20260806 22:37:41.774554 24592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15dd0c307c8840f59ccea5450ebc07e1. Latest consensus state: current_term: 1 leader_uuid: "15dd0c307c8840f59ccea5450ebc07e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15dd0c307c8840f59ccea5450ebc07e1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33419 } } }
I20260806 22:37:41.774791 24590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:41.774904 24592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:41.775377 24599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:41.775936 24599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:41.777974 24599 catalog_manager.cc:1347] Generated new cluster ID: f54dbfc0008340c9af2a5160bcee4f8b
I20260806 22:37:41.778038 24599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:41.793501 24599 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:41.794257 24599 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:41.801769 24599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15dd0c307c8840f59ccea5450ebc07e1: Generated new TSK 0
I20260806 22:37:41.802013 24599 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:37:41.829166 24624 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 22:37:41.829475 24624 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260806 22:37:41.829843 24623 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:41.812699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33419 (local address 127.0.0.1:47292)
0806 22:37:41.812921 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:41.812932 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:41.813029 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:41.813260 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:41.813261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:41.813264 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:37:41.813349 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:41.813352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.814067 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.814069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:41.814762 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:41.814765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.815206 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.815207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:41.815273 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:41.815297 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:37:41.829330 (+ 14033us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:37:41.829469 (+ 139us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.235.126:33419: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260806 22:37:41.830245 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:41.812815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.16.235.126:33419)
0806 22:37:41.812873 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:41.812875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:41.813003 (+ 128us) server_negotiation.cc:407] Connection header received
0806 22:37:41.813145 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:41.813146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:41.813160 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:41.813247 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:37:41.813427 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.813971 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.814867 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:41.815089 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:41.815167 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:41.815424 (+ 257us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:37:41.829138 (+ 13714us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:37:41.830154 (+ 1016us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47292: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 22:37:41.830300 24586 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47292: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.16.235.126:33419: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 22:37:41.830385 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 24496
2026-08-06T22:37:41Z chronyd exiting
I20260806 22:37:41.873363 17325 test_util.cc:175] -----------------------------------------------
I20260806 22:37:41.873473 17325 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786055836003438-17325-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (256 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 22:37:41.874320 17325 mini_oidc.cc:152] Starting JWKS server
I20260806 22:37:41.875129 17325 webserver.cc:460] Webserver started at https://127.0.0.1:39623/ using document root <none> and password file <none>
I20260806 22:37:41.875200 17325 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:37:41.875373 17325 webserver.cc:460] Webserver started at http://127.0.0.1:33123/ using document root <none> and password file <none>
2026-08-06T22:37:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:41Z Disabled control of system clock
I20260806 22:37:41.884755 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:43069
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:43069
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39623/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/testchainca.cert with env {}
W20260806 22:37:41.969359 24631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:41.969583 24631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:41.969609 24631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:41.970794 24631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:41.970842 24631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:41.970855 24631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:41.970868 24631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:41.970881 24631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:37:41.970894 24631 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:37:41.970906 24631 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39623/jwks.json
I20260806 22:37:41.972496 24631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:43069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:43069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39623/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:41.972713 24631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:41.972893 24631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:41.978281 24637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.978262 24638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:41.978295 24640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:41.978724 24631 server_base.cc:1034] running on GCE node
I20260806 22:37:41.978993 24631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:41.979355 24631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:41.980508 24631 hybrid_clock.cc:648] HybridClock initialized: now 1786055861980494 us; error 30 us; skew 500 ppm
I20260806 22:37:41.981859 24631 webserver.cc:460] Webserver started at http://127.16.235.126:33745/ using document root <none> and password file <none>
I20260806 22:37:41.982139 24631 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:41.982187 24631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:41.982372 24631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:41.983669 24631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "39ee95ef34e44fadbde42c0ede64dbdd"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "b03b9415f24282a009207150e3fc00b2"
server_key_iv: "627663bffef5b637ef07642fdc8f4c26"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.984062 24631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "39ee95ef34e44fadbde42c0ede64dbdd"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "b03b9415f24282a009207150e3fc00b2"
server_key_iv: "627663bffef5b637ef07642fdc8f4c26"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.985453 24631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:41.986255 24649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:41.986449 24631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:41.986557 24631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "39ee95ef34e44fadbde42c0ede64dbdd"
format_stamp: "Formatted at 2026-08-06 22:37:41 on dist-test-slave-3kk6"
server_key: "b03b9415f24282a009207150e3fc00b2"
server_key_iv: "627663bffef5b637ef07642fdc8f4c26"
server_key_version: "encryptionkey@0"
I20260806 22:37:41.986641 24631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:42.005048 24631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:42.005384 24631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:42.005502 24631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:42.009960 24631 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:43069
I20260806 22:37:42.009999 24711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:43069 every 8 connection(s)
I20260806 22:37:42.010418 24631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:42.011258 24712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:42.014305 24712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd: Bootstrap starting.
I20260806 22:37:42.015007 24712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:42.015327 24712 log.cc:824] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:42.016145 24712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd: No bootstrap required, opened a new log
I20260806 22:37:42.017815 24712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee95ef34e44fadbde42c0ede64dbdd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43069 } }
I20260806 22:37:42.017970 24712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:42.017994 24712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39ee95ef34e44fadbde42c0ede64dbdd, State: Initialized, Role: FOLLOWER
I20260806 22:37:42.018180 24712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee95ef34e44fadbde42c0ede64dbdd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43069 } }
I20260806 22:37:42.018247 24712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:42.018276 24712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:42.018354 24712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:42.018990 24712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee95ef34e44fadbde42c0ede64dbdd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43069 } }
I20260806 22:37:42.019115 24712 leader_election.cc:302] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39ee95ef34e44fadbde42c0ede64dbdd; no voters:
I20260806 22:37:42.019328 24712 leader_election.cc:288] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:42.019380 24718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:42.019564 24718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [term 1 LEADER]: Becoming Leader. State: Replica: 39ee95ef34e44fadbde42c0ede64dbdd, State: Running, Role: LEADER
I20260806 22:37:42.019631 24712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:42.019697 24718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee95ef34e44fadbde42c0ede64dbdd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43069 } }
I20260806 22:37:42.019959 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 24631
I20260806 22:37:42.020084 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:42.020219 24719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39ee95ef34e44fadbde42c0ede64dbdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee95ef34e44fadbde42c0ede64dbdd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43069 } } }
I20260806 22:37:42.020313 24719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:42.020316 17325 external_mini_cluster.cc:1428] Setting key 9a11be3fd868a88a230a5b7ac9d62a98
I20260806 22:37:42.021023 24720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39ee95ef34e44fadbde42c0ede64dbdd. Latest consensus state: current_term: 1 leader_uuid: "39ee95ef34e44fadbde42c0ede64dbdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee95ef34e44fadbde42c0ede64dbdd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43069 } } }
I20260806 22:37:42.021108 24720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd [sys.catalog]: This master's current role is: LEADER
W20260806 22:37:42.022240 24742 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:37:42.022300 24742 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:37:42.022362 24725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:42.022529 24725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:42.024303 24725 catalog_manager.cc:1347] Generated new cluster ID: 89269e5bc7ba486f973a48a2fdf2cd95
I20260806 22:37:42.024364 24725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:42.026803 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:42.021087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.16.235.126:43069)
0806 22:37:42.021292 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:42.021305 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:42.021333 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:37:42.021433 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:42.021439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:42.021493 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:42.021736 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:42.021882 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.023351 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.024499 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.025108 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.025219 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.026051 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:42.026056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:42.026056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:42.026269 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:42.026309 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:42.026323 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:42.026390 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:42.026490 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:37:42.026568 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260806 22:37:42.027573 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:37:42.034197 24725 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:42.034827 24725 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:42.042869 24725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39ee95ef34e44fadbde42c0ede64dbdd: Generated new TSK 0
I20260806 22:37:42.043092 24725 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:37:42.070210 24752 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 22:37:42.070334 24752 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:37:42.070735 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:42.048064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.16.235.126:43069)
0806 22:37:42.048115 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:42.048117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:42.048180 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:37:42.048282 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:42.048283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:42.048295 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:42.048355 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:37:42.048533 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.049028 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.050197 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.050425 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.050482 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.050748 (+ 266us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:37:42.070378 (+ 19630us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:37:42.070615 (+ 237us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40854: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:37:42.070848 24733 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40854: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 22:37:42.070952 24751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:42.047945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:43069 (local address 127.0.0.1:40854)
0806 22:37:42.048114 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:42.048127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:42.048202 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:42.048385 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:42.048386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:42.048388 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:37:42.048457 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:42.048459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.049118 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.049119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:42.050101 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:42.050104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.050513 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.050514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:42.050578 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.050584 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:37:42.070584 (+ 20000us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:37:42.070840 (+ 256us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.235.126:43069: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39623/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260806 22:37:42.071455 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 24631
2026-08-06T22:37:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (241 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 22:37:42.115902 17325 mini_oidc.cc:152] Starting JWKS server
I20260806 22:37:42.116618 17325 webserver.cc:460] Webserver started at https://127.0.0.1:41671/ using document root <none> and password file <none>
I20260806 22:37:42.116662 17325 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:37:42.116822 17325 webserver.cc:460] Webserver started at http://127.0.0.1:36171/ using document root <none> and password file <none>
2026-08-06T22:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:42Z Disabled control of system clock
I20260806 22:37:42.126180 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37469
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:37469
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41671/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:37:42.211109 24761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:42.211422 24761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:42.211467 24761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:42.212795 24761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:42.212879 24761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:42.212901 24761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:42.212924 24761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:42.212951 24761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:37:42.212983 24761 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:37:42.213009 24761 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41671/jwks.json
I20260806 22:37:42.214519 24761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:37469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41671/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:42.214879 24761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:42.215230 24761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:42.220656 24766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:42.220664 24771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:42.220651 24767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:42.221377 24761 server_base.cc:1034] running on GCE node
I20260806 22:37:42.221674 24761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:42.221971 24761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:42.223116 24761 hybrid_clock.cc:648] HybridClock initialized: now 1786055862223095 us; error 45 us; skew 500 ppm
I20260806 22:37:42.224488 24761 webserver.cc:460] Webserver started at http://127.16.235.126:37061/ using document root <none> and password file <none>
I20260806 22:37:42.224725 24761 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:42.224771 24761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:42.224872 24761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:42.226022 24761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "694b5b2806d44a56bbac87db2d813f41"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-3kk6"
server_key: "1739377c713bee03c92662175a372431"
server_key_iv: "2d711369aec2e0b2cca472f9c1ef83f0"
server_key_version: "encryptionkey@0"
I20260806 22:37:42.226397 24761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "694b5b2806d44a56bbac87db2d813f41"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-3kk6"
server_key: "1739377c713bee03c92662175a372431"
server_key_iv: "2d711369aec2e0b2cca472f9c1ef83f0"
server_key_version: "encryptionkey@0"
I20260806 22:37:42.227916 24761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:42.228741 24778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:42.228927 24761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:42.229031 24761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "694b5b2806d44a56bbac87db2d813f41"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-3kk6"
server_key: "1739377c713bee03c92662175a372431"
server_key_iv: "2d711369aec2e0b2cca472f9c1ef83f0"
server_key_version: "encryptionkey@0"
I20260806 22:37:42.229100 24761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:42.236271 24761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:42.236557 24761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:42.236670 24761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:42.240751 24761 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:37469
I20260806 22:37:42.240830 24850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:37469 every 8 connection(s)
I20260806 22:37:42.241189 24761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:42.241938 24852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:42.245064 24852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41: Bootstrap starting.
I20260806 22:37:42.245812 24852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:42.246133 24852 log.cc:824] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:42.247046 24852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41: No bootstrap required, opened a new log
I20260806 22:37:42.248703 24852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "694b5b2806d44a56bbac87db2d813f41" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37469 } }
I20260806 22:37:42.248886 24852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:42.248936 24852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 694b5b2806d44a56bbac87db2d813f41, State: Initialized, Role: FOLLOWER
I20260806 22:37:42.249114 24852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "694b5b2806d44a56bbac87db2d813f41" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37469 } }
I20260806 22:37:42.249204 24852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:42.249245 24852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:42.249302 24852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:42.249912 24852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "694b5b2806d44a56bbac87db2d813f41" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37469 } }
I20260806 22:37:42.250041 24852 leader_election.cc:302] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 694b5b2806d44a56bbac87db2d813f41; no voters:
I20260806 22:37:42.250280 24852 leader_election.cc:288] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:42.250466 24856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:42.250576 24852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:42.250634 24856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [term 1 LEADER]: Becoming Leader. State: Replica: 694b5b2806d44a56bbac87db2d813f41, State: Running, Role: LEADER
I20260806 22:37:42.250810 24856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "694b5b2806d44a56bbac87db2d813f41" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37469 } }
I20260806 22:37:42.251253 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 24761
I20260806 22:37:42.251286 24857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "694b5b2806d44a56bbac87db2d813f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "694b5b2806d44a56bbac87db2d813f41" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37469 } } }
I20260806 22:37:42.251376 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:42.251384 24857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:42.251601 17325 external_mini_cluster.cc:1428] Setting key 3d131d565b11c429e30c483d701d0e1b
I20260806 22:37:42.252250 24858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 694b5b2806d44a56bbac87db2d813f41. Latest consensus state: current_term: 1 leader_uuid: "694b5b2806d44a56bbac87db2d813f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "694b5b2806d44a56bbac87db2d813f41" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37469 } } }
I20260806 22:37:42.252341 24858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:42.252480 24868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:42.253062 24868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:42.255066 24868 catalog_manager.cc:1347] Generated new cluster ID: 1b7e444230694aa5a6ec613400c99747
I20260806 22:37:42.255129 24868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:42.258607 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:42.252590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.16.235.126:37469)
0806 22:37:42.252845 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:42.252854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:42.252890 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:37:42.253053 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:42.253061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:42.253198 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:42.253615 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:42.253711 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.255080 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.256266 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.256916 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.257054 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.257831 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:42.257836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:42.257836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:42.258064 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:42.258110 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:42.258125 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:42.258202 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:42.258277 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:37:42.258360 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":137,"threads_started":1}
I20260806 22:37:42.259578 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:37:42.265060 24868 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:42.265576 24868 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:42.271239 24868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 694b5b2806d44a56bbac87db2d813f41: Generated new TSK 0
I20260806 22:37:42.271462 24868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:37:42.280826 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:42.276757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49846 (local address 127.16.235.126:37469)
0806 22:37:42.276810 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:42.276812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:42.276893 (+ 81us) server_negotiation.cc:407] Connection header received
0806 22:37:42.276983 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:42.276984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:42.276999 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:42.277090 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:37:42.277256 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.278066 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.279017 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.279264 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.279355 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.279550 (+ 195us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:37:42.280554 (+ 1004us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:37:42.280715 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49846: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 22:37:42.280907 24873 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49846: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 22:37:42.280891 24892 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:42.276657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37469 (local address 127.0.0.1:49846)
0806 22:37:42.276811 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:42.276824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:42.276904 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:42.277103 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:42.277104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:42.277106 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:37:42.277183 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:42.277185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.278194 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.278195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:42.278926 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:42.278929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.279365 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.279366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:42.279432 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.279437 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:37:42.280666 (+ 1229us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:37:42.280783 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.235.126:37469: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":98,"threads_started":1}
I20260806 22:37:42.281200 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 24761
E20260806 22:37:42.297848 24893 webserver.cc:533] Webserver: sslize failed:
E20260806 22:37:42.297941 24893 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T22:37:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (210 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T22:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:42Z Disabled control of system clock
I20260806 22:37:42.335784 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37761
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:37761
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/testchainca.cert with env {}
W20260806 22:37:42.421429 24898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:42.421706 24898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:42.421747 24898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:42.423025 24898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:42.423112 24898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:42.423135 24898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:42.423156 24898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:42.423187 24898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:37:42.423213 24898 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:37:42.423243 24898 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 22:37:42.424880 24898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:37761
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:42.425202 24898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:42.425450 24898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:42.430719 24905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:42.430783 24908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:42.430984 24898 server_base.cc:1034] running on GCE node
W20260806 22:37:42.430755 24910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:42.431311 24898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:42.431624 24898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:42.432790 24898 hybrid_clock.cc:648] HybridClock initialized: now 1786055862432780 us; error 35 us; skew 500 ppm
I20260806 22:37:42.434244 24898 webserver.cc:460] Webserver started at http://127.16.235.126:39893/ using document root <none> and password file <none>
I20260806 22:37:42.434527 24898 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:42.434608 24898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:42.434794 24898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:42.435984 24898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "30b95e314e144f258cf9b3801dc9a0c0"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-3kk6"
server_key: "474d53b659dc52ba2d240f1ef17205dd"
server_key_iv: "7bf42683ac1c3932e26528e824174d8f"
server_key_version: "encryptionkey@0"
I20260806 22:37:42.436417 24898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "30b95e314e144f258cf9b3801dc9a0c0"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-3kk6"
server_key: "474d53b659dc52ba2d240f1ef17205dd"
server_key_iv: "7bf42683ac1c3932e26528e824174d8f"
server_key_version: "encryptionkey@0"
I20260806 22:37:42.437893 24898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:42.438623 24916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:42.438832 24898 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:37:42.438942 24898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "30b95e314e144f258cf9b3801dc9a0c0"
format_stamp: "Formatted at 2026-08-06 22:37:42 on dist-test-slave-3kk6"
server_key: "474d53b659dc52ba2d240f1ef17205dd"
server_key_iv: "7bf42683ac1c3932e26528e824174d8f"
server_key_version: "encryptionkey@0"
I20260806 22:37:42.439031 24898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:42.449915 24898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:42.450289 24898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:42.450462 24898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:42.454754 24898 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:37761
I20260806 22:37:42.454782 24982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:37761 every 8 connection(s)
I20260806 22:37:42.455189 24898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:42.456288 24984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:42.458868 24984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0: Bootstrap starting.
I20260806 22:37:42.459581 24984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:42.459905 24984 log.cc:824] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:42.460740 24984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0: No bootstrap required, opened a new log
I20260806 22:37:42.461002 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 24898
I20260806 22:37:42.461130 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:42.461398 17325 external_mini_cluster.cc:1428] Setting key 6d67799c73f67890070e2534db582ff7
I20260806 22:37:42.462992 24984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b95e314e144f258cf9b3801dc9a0c0" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37761 } }
I20260806 22:37:42.463168 24984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:42.463214 24984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30b95e314e144f258cf9b3801dc9a0c0, State: Initialized, Role: FOLLOWER
I20260806 22:37:42.463380 24984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b95e314e144f258cf9b3801dc9a0c0" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37761 } }
I20260806 22:37:42.463480 24984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:42.463527 24984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:42.463581 24984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:42.464251 24984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b95e314e144f258cf9b3801dc9a0c0" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37761 } }
I20260806 22:37:42.464386 24984 leader_election.cc:302] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30b95e314e144f258cf9b3801dc9a0c0; no voters:
I20260806 22:37:42.464594 24984 leader_election.cc:288] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:42.464630 24994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:42.464744 24994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [term 1 LEADER]: Becoming Leader. State: Replica: 30b95e314e144f258cf9b3801dc9a0c0, State: Running, Role: LEADER
I20260806 22:37:42.464887 24984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:42.464870 24994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b95e314e144f258cf9b3801dc9a0c0" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37761 } }
I20260806 22:37:42.465216 24998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30b95e314e144f258cf9b3801dc9a0c0. Latest consensus state: current_term: 1 leader_uuid: "30b95e314e144f258cf9b3801dc9a0c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b95e314e144f258cf9b3801dc9a0c0" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37761 } } }
I20260806 22:37:42.465312 24998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:42.465504 24995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30b95e314e144f258cf9b3801dc9a0c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30b95e314e144f258cf9b3801dc9a0c0" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37761 } } }
I20260806 22:37:42.465569 24995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:42.465665 25001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:42.466246 25001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:42.468532 25001 catalog_manager.cc:1347] Generated new cluster ID: 5e461a6c5f364b4297e471fbdda594d8
I20260806 22:37:42.468591 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:42.462060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36004 (local address 127.16.235.126:37761)
0806 22:37:42.462321 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:42.462329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:42.462356 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:37:42.462924 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:42.462932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:42.463013 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:42.463166 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:42.463378 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.464920 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.466050 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.466800 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.467275 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.467849 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:42.467860 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:42.467861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:37:42.468040 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:42.468073 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:37:42.468084 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:42.468146 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:42.468295 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:37:42.468378 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":115,"threads_started":1}
I20260806 22:37:42.468591 25001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:42.471679 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:37:42.479511 25001 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:42.480034 25001 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:42.489939 25001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30b95e314e144f258cf9b3801dc9a0c0: Generated new TSK 0
I20260806 22:37:42.490195 25001 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:37:42.622332 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:42.503395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36016 (local address 127.16.235.126:37761)
0806 22:37:42.503468 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:42.503470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:42.503612 (+ 142us) server_negotiation.cc:407] Connection header received
0806 22:37:42.503694 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:42.503695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:42.503707 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:42.503774 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:37:42.503933 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.504436 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.505568 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.505766 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.505828 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.506021 (+ 193us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:37:42.622075 (+116054us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:37:42.622266 (+ 191us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36016: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 22:37:42.622444 24993 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36016: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 22:37:42.622539 25025 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:42.503333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37761 (local address 127.0.0.1:36016)
0806 22:37:42.503532 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:42.503545 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:42.503617 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:42.503793 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:42.503794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:42.503796 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:37:42.503864 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:42.503867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.504517 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.504518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:42.505475 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:42.505478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:42.505853 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:42.505854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:42.505914 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:42.505920 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:37:42.622250 (+116330us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:37:42.622423 (+ 173us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.235.126:37761: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":119,"threads_started":1}
I20260806 22:37:42.622995 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 24898
2026-08-06T22:37:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (320 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (98 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:42 dist-test-slave-3kk6 krb5kdc[25043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:42 dist-test-slave-3kk6 krb5kdc[25043](info): set up 2 sockets
Aug 06 22:37:42 dist-test-slave-3kk6 krb5kdc[25043](info): commencing operation
krb5kdc: starting...
W20260806 22:37:44.058765 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:44.193841 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36323
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:32901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:36323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:44.279313 25056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:44.279547 25056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:44.279572 25056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:44.280795 25056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:44.280843 25056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:44.280858 25056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:44.280871 25056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:44.280929 25056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:44.282436 25056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:32901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:36323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:44.282758 25056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:44.283018 25056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:44.288241 25065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:44.288276 25063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:44.288282 25056 server_base.cc:1034] running on GCE node
W20260806 22:37:44.288241 25068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:44.288741 25056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:44.289058 25056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:44.290207 25056 hybrid_clock.cc:648] HybridClock initialized: now 1786055864290184 us; error 42 us; skew 500 ppm
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.292928 25056 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:44.293370 25056 webserver.cc:460] Webserver started at http://127.16.235.126:38881/ using document root <none> and password file <none>
I20260806 22:37:44.293607 25056 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:44.293680 25056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:44.293843 25056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:44.295032 25056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "5da55df69e6f4d0981401509be83212d"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "d634e56e3c3094e212d4badf0b87ba6a"
server_key_iv: "171fbf6ce6e70ee26b6bbb8f48416637"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.295423 25056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "5da55df69e6f4d0981401509be83212d"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "d634e56e3c3094e212d4badf0b87ba6a"
server_key_iv: "171fbf6ce6e70ee26b6bbb8f48416637"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.296936 25056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:44.297703 25076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.297879 25056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:44.297992 25056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "5da55df69e6f4d0981401509be83212d"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "d634e56e3c3094e212d4badf0b87ba6a"
server_key_iv: "171fbf6ce6e70ee26b6bbb8f48416637"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.298074 25056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:44.306394 25056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:44.308451 25056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:44.308629 25056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:44.313021 25056 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:36323
I20260806 22:37:44.313077 25143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:36323 every 8 connection(s)
I20260806 22:37:44.313467 25056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:44.314419 25145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:44.316847 25145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d: Bootstrap starting.
I20260806 22:37:44.317415 25145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:44.317678 25145 log.cc:824] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:44.318378 25145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d: No bootstrap required, opened a new log
I20260806 22:37:44.319885 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 25056
I20260806 22:37:44.320011 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:44.320221 17325 external_mini_cluster.cc:1428] Setting key fc1ecf44161abec838fe90f521ad9040
I20260806 22:37:44.320165 25145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da55df69e6f4d0981401509be83212d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36323 } }
I20260806 22:37:44.320341 25145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:44.320411 25145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5da55df69e6f4d0981401509be83212d, State: Initialized, Role: FOLLOWER
I20260806 22:37:44.320575 25145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da55df69e6f4d0981401509be83212d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36323 } }
I20260806 22:37:44.320680 25145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:44.320740 25145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:44.320807 25145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:44.322135 25145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da55df69e6f4d0981401509be83212d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36323 } }
I20260806 22:37:44.322295 25145 leader_election.cc:302] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5da55df69e6f4d0981401509be83212d; no voters:
I20260806 22:37:44.322489 25145 leader_election.cc:288] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:44.322540 25155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:44.322695 25155 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [term 1 LEADER]: Becoming Leader. State: Replica: 5da55df69e6f4d0981401509be83212d, State: Running, Role: LEADER
I20260806 22:37:44.322819 25145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:44.322808 25155 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da55df69e6f4d0981401509be83212d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36323 } }
I20260806 22:37:44.323153 25155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5da55df69e6f4d0981401509be83212d. Latest consensus state: current_term: 1 leader_uuid: "5da55df69e6f4d0981401509be83212d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da55df69e6f4d0981401509be83212d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36323 } } }
I20260806 22:37:44.323225 25155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:44.323176 25156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5da55df69e6f4d0981401509be83212d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da55df69e6f4d0981401509be83212d" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36323 } } }
I20260806 22:37:44.323421 25156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:44.323427 25161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:44.324081 25161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:44.326056 25161 catalog_manager.cc:1347] Generated new cluster ID: 1c58ba5a92864cfcb389f611be21e24f
I20260806 22:37:44.326116 25161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:44.331950 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.320809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.16.235.126:36323)
0806 22:37:44.320985 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.320993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.321046 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:37:44.321173 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.321178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.321231 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.321344 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.322294 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.323764 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.324891 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.325507 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.325615 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.328191 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:44.328199 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:44.328200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:44.328211 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:44.329968 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.330606 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.330610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.330610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.330792 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.331032 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.331033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.331033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.331310 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:44.331534 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:44.331606 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:37:44.331780 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260806 22:37:44.349907 25161 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:44.350521 25161 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:44.352695 25161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5da55df69e6f4d0981401509be83212d: Generated new TSK 0
I20260806 22:37:44.352936 25161 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:44.396001 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36323
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:32901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:44.496647 25182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:44.496879 25182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:44.496905 25182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:44.498018 25182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:44.498056 25182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:44.498082 25182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:44.499684 25182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:32901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:44.499884 25182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:44.500089 25182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:44.505509 25195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:44.505535 25193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:44.505601 25182 server_base.cc:1034] running on GCE node
W20260806 22:37:44.505509 25198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:44.506105 25182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:44.506336 25182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:44.507539 25182 hybrid_clock.cc:648] HybridClock initialized: now 1786055864507518 us; error 50 us; skew 500 ppm
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.510481 25182 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:44.511047 25182 webserver.cc:460] Webserver started at http://127.16.235.65:32971/ using document root <none> and password file <none>
I20260806 22:37:44.511317 25182 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:44.511364 25182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:44.511533 25182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:44.512676 25182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "d70c383f349843058e5f0698cc067a56"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "89f38e4b8bea42d188377ac54e0e89d0"
server_key_iv: "06e2e173a2a82b377aa03e3021cbd2f7"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.513070 25182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "d70c383f349843058e5f0698cc067a56"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "89f38e4b8bea42d188377ac54e0e89d0"
server_key_iv: "06e2e173a2a82b377aa03e3021cbd2f7"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.514488 25182 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:44.515379 25205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.515568 25182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:44.515676 25182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "d70c383f349843058e5f0698cc067a56"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "89f38e4b8bea42d188377ac54e0e89d0"
server_key_iv: "06e2e173a2a82b377aa03e3021cbd2f7"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.515763 25182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:44.540438 25182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:44.542891 25182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:44.543099 25182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:44.543401 25182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:44.543741 25182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:44.543797 25182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.543840 25182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:44.543865 25182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.550586 25182 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:38847
I20260806 22:37:44.550634 25355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:38847 every 8 connection(s)
I20260806 22:37:44.551023 25182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:44.553472 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 25182
I20260806 22:37:44.553638 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:44.553858 17325 external_mini_cluster.cc:1428] Setting key a3d9a461a1c068fba21d50ef6424a3fa
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:44.562515 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.551903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:49671 (local address 127.16.235.126:36323)
0806 22:37:44.551953 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.551955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.552177 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:37:44.553238 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.553240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.553260 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.553318 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.554213 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.554978 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.556047 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.556392 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.556475 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.559250 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:44.559251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:44.559252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:44.559263 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:44.560949 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.561458 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.561459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.561459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.561529 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.561804 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.561806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.561806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.561940 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:44.562021 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:44.562319 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:37:44.562421 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:37:44.562762 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.551854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36323 (local address 127.16.235.65:49671)
0806 22:37:44.552040 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:44.552085 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:44.553098 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:44.553402 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:44.553406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:44.553783 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:44.554068 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:44.554075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.555098 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.555099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:44.555942 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:44.555945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.556522 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.556523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:44.556697 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.557340 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:44.557353 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:44.559130 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:44.561074 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:44.561082 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:44.561085 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:44.561352 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:44.561604 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:44.561605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:44.561605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:44.561726 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:44.562012 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:44.562019 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:44.562175 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 22:37:44.562295 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 22:37:44.562474 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260806 22:37:44.563402 25356 heartbeater.cc:347] Connected to a master server at 127.16.235.126:36323
I20260806 22:37:44.563508 25356 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
I20260806 22:37:44.563722 25356 heartbeater.cc:511] Master 127.16.235.126:36323 requested a full tablet report, sending...
I20260806 22:37:44.564337 25101 ts_manager.cc:194] Registered new tserver with Master: d70c383f349843058e5f0698cc067a56 (127.16.235.65:38847)
I20260806 22:37:44.565025 25101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:49671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:44.583530 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36323
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:32901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:44.668567 25362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:44.668802 25362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:44.668825 25362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:44.669994 25362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:44.670042 25362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:44.670068 25362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:44.671630 25362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:32901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:44.671907 25362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:44.672151 25362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:44.677484 25373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:44.677435 25376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:44.677549 25362 server_base.cc:1034] running on GCE node
W20260806 22:37:44.677435 25370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:44.677886 25362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:44.678154 25362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:44.679298 25362 hybrid_clock.cc:648] HybridClock initialized: now 1786055864679283 us; error 34 us; skew 500 ppm
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.681821 25362 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:44.682214 25362 webserver.cc:460] Webserver started at http://127.16.235.66:39779/ using document root <none> and password file <none>
I20260806 22:37:44.682415 25362 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:44.682456 25362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:44.682548 25362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:44.683672 25362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "6b88e167e15649c5acf2524d0d683acd"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "d9edaa55eaca59ad7a0e8d187df5047d"
server_key_iv: "e851f1dad097db7c6bdf129ec4aa885d"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.683995 25362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "6b88e167e15649c5acf2524d0d683acd"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "d9edaa55eaca59ad7a0e8d187df5047d"
server_key_iv: "e851f1dad097db7c6bdf129ec4aa885d"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.685350 25362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:44.686100 25386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.686275 25362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:44.686388 25362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "6b88e167e15649c5acf2524d0d683acd"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "d9edaa55eaca59ad7a0e8d187df5047d"
server_key_iv: "e851f1dad097db7c6bdf129ec4aa885d"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.686470 25362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:44.704344 25362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:44.706362 25362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:44.706552 25362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:44.706903 25362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:44.707291 25362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:44.707353 25362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.707399 25362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:44.707428 25362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.713689 25362 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:42019
I20260806 22:37:44.713786 25542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:42019 every 8 connection(s)
I20260806 22:37:44.714154 25362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:44.719867 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 25362
I20260806 22:37:44.720002 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:44.720225 17325 external_mini_cluster.cc:1428] Setting key f3c7807fc0e073875024a73257df2e57
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:44.725435 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.715001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36323 (local address 127.16.235.66:41409)
0806 22:37:44.715224 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:44.715272 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:44.716222 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:44.716459 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:44.716462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:44.716755 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:44.716965 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:44.716969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.717637 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.717637 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:44.718447 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:44.718449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.718951 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.718952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:44.719058 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.719550 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:44.719558 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:44.721510 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:44.723629 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:44.723638 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:44.723641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:44.724002 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:44.724255 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:44.724256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:44.724256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:44.724422 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:44.724828 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:44.724835 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:44.725010 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 22:37:44.725105 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:37:44.725245 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":127,"threads_started":1}
I20260806 22:37:44.725441 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.715016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:41409 (local address 127.16.235.126:36323)
0806 22:37:44.715091 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.715093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.715371 (+ 278us) server_negotiation.cc:407] Connection header received
0806 22:37:44.716332 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.716333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.716345 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.716401 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.717047 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.717550 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.718550 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.718872 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.718932 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.721610 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:44.721612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:44.721612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:44.721621 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:44.723526 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.724092 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.724094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.724094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.724159 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.724558 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.724560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.724561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.724742 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:44.724835 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:44.725161 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:37:44.725289 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:37:44.725947 25544 heartbeater.cc:347] Connected to a master server at 127.16.235.126:36323
I20260806 22:37:44.726069 25544 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:44.726316 25544 heartbeater.cc:511] Master 127.16.235.126:36323 requested a full tablet report, sending...
I20260806 22:37:44.726823 25101 ts_manager.cc:194] Registered new tserver with Master: 6b88e167e15649c5acf2524d0d683acd (127.16.235.66:42019)
I20260806 22:37:44.727212 25101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:41409
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:44.751626 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36323
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:32901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:44.838828 25550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:44.839051 25550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:44.839076 25550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:44.840267 25550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:44.840317 25550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:44.840343 25550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:44.841961 25550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:32901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:44.842229 25550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:44.842494 25550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:44.848452 25556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:44.848819 25550 server_base.cc:1034] running on GCE node
W20260806 22:37:44.848469 25560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:44.848469 25557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:44.849428 25550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:44.849814 25550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:44.851011 25550 hybrid_clock.cc:648] HybridClock initialized: now 1786055864850980 us; error 56 us; skew 500 ppm
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.854632 25550 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:44.855419 25550 webserver.cc:460] Webserver started at http://127.16.235.67:33837/ using document root <none> and password file <none>
I20260806 22:37:44.855897 25550 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:44.855973 25550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:44.856132 25550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:44.857666 25550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "530f3ef7d0ec42508b3696ed27478cdf"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "6e9715aaea87172cf0960df955f375f3"
server_key_iv: "1cc3c6136ebd9f8ab9605d00c54fc899"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.858139 25550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "530f3ef7d0ec42508b3696ed27478cdf"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "6e9715aaea87172cf0960df955f375f3"
server_key_iv: "1cc3c6136ebd9f8ab9605d00c54fc899"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.860069 25550 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.002s
I20260806 22:37:44.861433 25568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.861989 25550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:44.862164 25550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "530f3ef7d0ec42508b3696ed27478cdf"
format_stamp: "Formatted at 2026-08-06 22:37:44 on dist-test-slave-3kk6"
server_key: "6e9715aaea87172cf0960df955f375f3"
server_key_iv: "1cc3c6136ebd9f8ab9605d00c54fc899"
server_key_version: "encryptionkey@0"
I20260806 22:37:44.862282 25550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:44.881647 25550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:44.885951 25550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:44.886224 25550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:44.886723 25550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:44.887291 25550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:44.887390 25550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.887497 25550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:44.887542 25550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:44.899082 25550 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:38903
I20260806 22:37:44.899287 25720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:38903 every 8 connection(s)
I20260806 22:37:44.900475 25550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:44.908805 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 25550
I20260806 22:37:44.908908 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:44.909224 17325 external_mini_cluster.cc:1428] Setting key 44bd3f80c0ad3d06dabc27d37fd95fd9
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:44.916116 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.902301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:44443 (local address 127.16.235.126:36323)
0806 22:37:44.902379 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.902381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.902898 (+ 517us) server_negotiation.cc:407] Connection header received
0806 22:37:44.905020 (+ 2122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.905022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.905050 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.905121 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.906105 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.907180 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.908612 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.908909 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.909030 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.912885 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:44.912887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:44.912887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:44.912895 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:44.914778 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.915209 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.915211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.915211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.915289 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.915540 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.915541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.915541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.915662 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:44.915717 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:44.915974 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 22:37:44.916047 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:37:44.916303 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.902285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36323 (local address 127.16.235.67:44443)
0806 22:37:44.902492 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:44.902563 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:44.904872 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:44.905224 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:44.905228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:44.905648 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:44.905961 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:44.905967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.907371 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.907372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:44.908458 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:44.908462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.909159 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.909161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:44.909574 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.910560 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:44.910572 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:44.912737 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:44.914862 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:44.914868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:44.914870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:44.915103 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:44.915361 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:44.915361 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:44.915362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:44.915475 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:44.915720 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:44.915724 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:44.915856 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 22:37:44.915944 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:37:44.916074 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260806 22:37:44.916824 25722 heartbeater.cc:347] Connected to a master server at 127.16.235.126:36323
I20260806 22:37:44.916960 25722 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:44.917177 25722 heartbeater.cc:511] Master 127.16.235.126:36323 requested a full tablet report, sending...
I20260806 22:37:44.917600 25101 ts_manager.cc:194] Registered new tserver with Master: 530f3ef7d0ec42508b3696ed27478cdf (127.16.235.67:38903)
I20260806 22:37:44.917994 25101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:44443
I20260806 22:37:44.922175 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/test-admin.keytab.
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.948890 17325 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 22:37:44.951620 25730 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:44.950342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54854 (local address 127.16.235.65:38847)
0806 22:37:44.950608 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.950613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.950626 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:37:44.950723 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.950729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.950754 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.950867 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.951500 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54854: BlockingRecv error: recv got EOF from 127.0.0.1:54854 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":150,"threads_started":1}
W20260806 22:37:44.951671 17325 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.953025 17325 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.954232 17325 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:44.964249 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.954978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54856 (local address 127.16.235.65:38847)
0806 22:37:44.955079 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.955081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.955189 (+ 108us) server_negotiation.cc:407] Connection header received
0806 22:37:44.955283 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.955284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.955297 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.955351 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.955925 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.956919 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.957790 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.958266 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.958408 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.961204 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:44.961215 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:44.961216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:44.961225 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:44.963007 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.963365 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.963368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.963369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.963425 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.963594 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.963595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.963595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.963757 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:44.963925 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:44.963970 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 22:37:44.964054 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 22:37:44.967319 25730 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:44.966461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54868 (local address 127.16.235.65:38847)
0806 22:37:44.966574 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.966575 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.966745 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:37:44.966777 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.966778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.966788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.966873 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.967227 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54868: BlockingRecv error: recv got EOF from 127.0.0.1:54868 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 22:37:44.967427 17325 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.968945 17325 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.970103 17325 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:44.979736 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.970813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.16.235.65:38847)
0806 22:37:44.970898 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.970900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.970958 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:37:44.971031 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.971032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.971045 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.971103 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.971704 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.972246 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.973146 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.973532 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.973747 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.976709 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:44.976710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:44.976711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:44.976718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:44.978280 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.978786 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.978789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.978789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.978867 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.979145 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.979146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.979147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.979268 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:44.979386 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:44.979494 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:37:44.979605 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 22:37:44.982240 25730 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:44.981565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54888 (local address 127.16.235.65:38847)
0806 22:37:44.981622 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.981624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.981699 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:37:44.981740 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.981741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.981752 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.981822 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.982179 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54888: BlockingRecv error: recv got EOF from 127.0.0.1:54888 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:37:44.982345 17325 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786055836003438-17325-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.983855 17325 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:44.985244 17325 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:37:44 dist-test-slave-3kk6 krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:37:44.994910 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:44.985894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.16.235.65:38847)
0806 22:37:44.985945 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:44.985947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:44.986116 (+ 169us) server_negotiation.cc:407] Connection header received
0806 22:37:44.986156 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:44.986157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:44.986169 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:44.986228 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:44.986634 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.987333 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.988318 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:44.988575 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:44.988705 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:44.991974 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:44.991975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:44.991976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:44.991984 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:44.993565 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.994001 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.994002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.994002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.994064 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:44.994276 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:44.994276 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:44.994277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:44.994398 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:44.994526 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:44.994630 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:37:44.994782 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:37:44.995950 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 25182
I20260806 22:37:45.015714 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 25362
I20260806 22:37:45.039796 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 25550
I20260806 22:37:45.065575 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 25056
2026-08-06T22:37:45Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2255 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:45 dist-test-slave-3kk6 krb5kdc[25752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:45 dist-test-slave-3kk6 krb5kdc[25752](info): set up 2 sockets
Aug 06 22:37:45 dist-test-slave-3kk6 krb5kdc[25752](info): commencing operation
krb5kdc: starting...
W20260806 22:37:46.400825 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:46 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:46Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.126.keytab.
Entry for principal oryx/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.126.keytab.
I20260806 22:37:46.531163 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:44375
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.126.keytab
--principal=oryx/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:44375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:46.630290 25771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:46.630573 25771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:46.630612 25771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:46.631842 25771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:46.631927 25771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:46.631950 25771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:46.631973 25771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:46.632001 25771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:46.633590 25771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:44375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.126.keytab
--principal=oryx/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:44375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:46.633903 25771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:46.634155 25771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:46.639459 25778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:46.639428 25780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:46.639428 25776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:46.639886 25771 server_base.cc:1034] running on GCE node
I20260806 22:37:46.640170 25771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:46.640511 25771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:46.641703 25771 hybrid_clock.cc:648] HybridClock initialized: now 1786055866641682 us; error 53 us; skew 500 ppm
Aug 06 22:37:46 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:46.651940 25771 init.cc:377] Logged in from keytab as oryx/127.16.235.126@KRBTEST.COM (short username oryx)
I20260806 22:37:46.652491 25771 webserver.cc:460] Webserver started at http://127.16.235.126:38489/ using document root <none> and password file <none>
I20260806 22:37:46.652756 25771 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:46.652833 25771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:46.653013 25771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:46.654217 25771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "047a87bf14f8429686f8d19bb1ec9fb6"
format_stamp: "Formatted at 2026-08-06 22:37:46 on dist-test-slave-3kk6"
server_key: "3210e7ccab539893de0c18f7b99a5273"
server_key_iv: "9d6e72f331a4a9d278377c14e3a7eab3"
server_key_version: "encryptionkey@0"
I20260806 22:37:46.654645 25771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "047a87bf14f8429686f8d19bb1ec9fb6"
format_stamp: "Formatted at 2026-08-06 22:37:46 on dist-test-slave-3kk6"
server_key: "3210e7ccab539893de0c18f7b99a5273"
server_key_iv: "9d6e72f331a4a9d278377c14e3a7eab3"
server_key_version: "encryptionkey@0"
I20260806 22:37:46.656270 25771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:46.657235 25789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.657522 25771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:46.657634 25771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "047a87bf14f8429686f8d19bb1ec9fb6"
format_stamp: "Formatted at 2026-08-06 22:37:46 on dist-test-slave-3kk6"
server_key: "3210e7ccab539893de0c18f7b99a5273"
server_key_iv: "9d6e72f331a4a9d278377c14e3a7eab3"
server_key_version: "encryptionkey@0"
I20260806 22:37:46.657729 25771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:46.677184 25771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:46.679678 25771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:46.679885 25771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:46.684437 25771 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:44375
I20260806 22:37:46.684480 25858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:44375 every 8 connection(s)
I20260806 22:37:46.684903 25771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:46.686035 25860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:46.688057 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 25771
I20260806 22:37:46.688189 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:46.688436 17325 external_mini_cluster.cc:1428] Setting key 183acde68179b2b9f42632dd93b07859
I20260806 22:37:46.688792 25860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: Bootstrap starting.
I20260806 22:37:46.689911 25860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:46.690315 25860 log.cc:824] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:46.691231 25860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: No bootstrap required, opened a new log
I20260806 22:37:46.693706 25860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } }
I20260806 22:37:46.693897 25860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:46.693945 25860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047a87bf14f8429686f8d19bb1ec9fb6, State: Initialized, Role: FOLLOWER
I20260806 22:37:46.694116 25860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } }
I20260806 22:37:46.694263 25860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:46.694626 25860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:46.694775 25860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:46.695468 25860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } }
I20260806 22:37:46.695600 25860 leader_election.cc:302] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 047a87bf14f8429686f8d19bb1ec9fb6; no voters:
I20260806 22:37:46.695827 25860 leader_election.cc:288] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:46.695902 25872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:46.696105 25860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:46.696110 25872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 1 LEADER]: Becoming Leader. State: Replica: 047a87bf14f8429686f8d19bb1ec9fb6, State: Running, Role: LEADER
Aug 06 22:37:46 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:46.696312 25872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } }
I20260806 22:37:46.696700 25875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } }
I20260806 22:37:46.696682 25877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 047a87bf14f8429686f8d19bb1ec9fb6. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } }
I20260806 22:37:46.696780 25875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:46.696784 25877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:46.697589 25888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:46.698196 25888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:46.700201 25888 catalog_manager.cc:1347] Generated new cluster ID: dbff21308e2f421bbbd3035d967e73e1
I20260806 22:37:46.700263 25888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:46.701145 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.689025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.16.235.126:44375)
0806 22:37:46.689231 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:46.689242 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:46.689282 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:37:46.689961 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:46.689972 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:46.690051 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:46.690248 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:46.691091 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.692549 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.693320 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.693974 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.694089 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.697039 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:46.697044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:46.697044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:46.697058 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:46.699354 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.699827 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.699832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.699838 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.699927 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.700172 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.700173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.700173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.700491 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:46.700660 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:46.700784 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 22:37:46.700932 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
I20260806 22:37:46.731523 25888 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:46.732144 25888 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:46.736896 25888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: Generated new TSK 0
I20260806 22:37:46.737200 25888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab.
Entry for principal oryx/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab.
I20260806 22:37:46.767305 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:44375
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab
--principal=oryx/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:46.855825 25904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:46.856096 25904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:46.856141 25904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:46.857365 25904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:46.857438 25904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:46.857484 25904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:46.859149 25904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab
--principal=oryx/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:44375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:46.859462 25904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:46.859735 25904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:46.865440 25910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:46.865446 25911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:46.865437 25916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:46.865705 25904 server_base.cc:1034] running on GCE node
I20260806 22:37:46.865906 25904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:46.866209 25904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:46.867393 25904 hybrid_clock.cc:648] HybridClock initialized: now 1786055866867367 us; error 56 us; skew 500 ppm
Aug 06 22:37:46 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:46.870119 25904 init.cc:377] Logged in from keytab as oryx/127.16.235.65@KRBTEST.COM (short username oryx)
I20260806 22:37:46.870558 25904 webserver.cc:460] Webserver started at http://127.16.235.65:41347/ using document root <none> and password file <none>
I20260806 22:37:46.870850 25904 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:46.870903 25904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:46.871042 25904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:46.872195 25904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "6bcee1edca234af1845705222b536dff"
format_stamp: "Formatted at 2026-08-06 22:37:46 on dist-test-slave-3kk6"
server_key: "1aa7865e9a9dc396f23f6945cc20608a"
server_key_iv: "4d1a9370ee62cb85649fc473dc9ffef7"
server_key_version: "encryptionkey@0"
I20260806 22:37:46.872596 25904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "6bcee1edca234af1845705222b536dff"
format_stamp: "Formatted at 2026-08-06 22:37:46 on dist-test-slave-3kk6"
server_key: "1aa7865e9a9dc396f23f6945cc20608a"
server_key_iv: "4d1a9370ee62cb85649fc473dc9ffef7"
server_key_version: "encryptionkey@0"
I20260806 22:37:46.874105 25904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:46.875094 25928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.875312 25904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:46.875433 25904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "6bcee1edca234af1845705222b536dff"
format_stamp: "Formatted at 2026-08-06 22:37:46 on dist-test-slave-3kk6"
server_key: "1aa7865e9a9dc396f23f6945cc20608a"
server_key_iv: "4d1a9370ee62cb85649fc473dc9ffef7"
server_key_version: "encryptionkey@0"
I20260806 22:37:46.875519 25904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:46.905412 25904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:46.907577 25904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:46.907765 25904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:46.908057 25904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:46.908434 25904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:46.908504 25904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.908552 25904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:46.908576 25904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:46.915416 25904 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:42679
I20260806 22:37:46.915462 26079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:42679 every 8 connection(s)
I20260806 22:37:46.915833 25904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:37:46 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:46.924479 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 25904
I20260806 22:37:46.924582 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:46.924800 17325 external_mini_cluster.cc:1428] Setting key 308dac74b0b7e9bcd815436fe60a4aa0
I20260806 22:37:46.926317 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.916719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:50995 (local address 127.16.235.126:44375)
0806 22:37:46.916769 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:46.916772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:46.917034 (+ 262us) server_negotiation.cc:407] Connection header received
0806 22:37:46.917624 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:46.917625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:46.917639 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:46.917703 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:46.918881 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.919420 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.920474 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.920695 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.920811 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.923231 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:46.923233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:46.923233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:46.923242 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:46.924735 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.925248 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.925249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.925250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.925321 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:46.925603 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:46.925604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:46.925604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:46.925737 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:46.925813 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:46.926121 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 22:37:46.926223 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:37:46.926559 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:46.916693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.65:50995)
0806 22:37:46.916897 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:46.916938 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:46.917507 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:46.917775 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:46.917780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:46.918453 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:46.918734 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.918741 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.919536 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.919537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.920375 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:46.920378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:46.920826 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:46.920827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:46.920967 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:46.921524 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:46.921533 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:46.923120 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:46.924829 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:46.924838 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:46.924841 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:46.925142 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:46.925403 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:46.925404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:46.925404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:46.925525 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:46.925815 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:46.925824 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:46.925978 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 22:37:46.926087 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:37:46.926278 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":106,"threads_started":1}
I20260806 22:37:46.927112 26080 heartbeater.cc:347] Connected to a master server at 127.16.235.126:44375
I20260806 22:37:46.927237 26080 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:46.927470 26080 heartbeater.cc:511] Master 127.16.235.126:44375 requested a full tablet report, sending...
I20260806 22:37:46.928118 25810 ts_manager.cc:194] Registered new tserver with Master: 6bcee1edca234af1845705222b536dff (127.16.235.65:42679)
I20260806 22:37:46.928843 25810 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:50995
WARNING: no policy specified for oryx/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab.
Entry for principal oryx/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab.
I20260806 22:37:46.954169 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:44375
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab
--principal=oryx/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:47.041388 26087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:47.041612 26087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:47.041638 26087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:47.042780 26087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:47.042817 26087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:47.042843 26087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:47.044268 26087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab
--principal=oryx/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:44375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:47.044484 26087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:47.044678 26087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:47.050166 26093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.050189 26095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:47.050346 26087 server_base.cc:1034] running on GCE node
W20260806 22:37:47.050203 26092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:47.050772 26087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:47.051017 26087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:47.052189 26087 hybrid_clock.cc:648] HybridClock initialized: now 1786055867052177 us; error 34 us; skew 500 ppm
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:47.055011 26087 init.cc:377] Logged in from keytab as oryx/127.16.235.66@KRBTEST.COM (short username oryx)
I20260806 22:37:47.055459 26087 webserver.cc:460] Webserver started at http://127.16.235.66:36165/ using document root <none> and password file <none>
I20260806 22:37:47.055722 26087 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:47.055774 26087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:47.055931 26087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:47.057062 26087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "83c6bb1251604c9dad9f23e814657bcf"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "b31bfc6d0241d1d307c7586f6b505716"
server_key_iv: "9bb1f1cc7aefe65fc53cee2ebbb7ad4f"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.057479 26087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "83c6bb1251604c9dad9f23e814657bcf"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "b31bfc6d0241d1d307c7586f6b505716"
server_key_iv: "9bb1f1cc7aefe65fc53cee2ebbb7ad4f"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.059085 26087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:47.059950 26108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.060220 26087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:47.060346 26087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "83c6bb1251604c9dad9f23e814657bcf"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "b31bfc6d0241d1d307c7586f6b505716"
server_key_iv: "9bb1f1cc7aefe65fc53cee2ebbb7ad4f"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.060439 26087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:47.077565 26087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:47.079938 26087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:47.080109 26087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:47.080397 26087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:47.080762 26087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:47.080821 26087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.080859 26087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:47.080888 26087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.087126 26087 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:42963
I20260806 22:37:47.087174 26263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:42963 every 8 connection(s)
I20260806 22:37:47.087565 26087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:47.090931 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 26087
I20260806 22:37:47.091038 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:47.091281 17325 external_mini_cluster.cc:1428] Setting key 9931d647286bfbf92ded7245417a7d3c
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:47.099805 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.088420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:45641 (local address 127.16.235.126:44375)
0806 22:37:47.088512 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.088515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.088644 (+ 129us) server_negotiation.cc:407] Connection header received
0806 22:37:47.089239 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.089240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.089252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.089316 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.090332 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.090973 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.092583 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.092780 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.092845 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.096003 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.096005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.096005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.096014 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.097943 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.098545 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.098546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.098547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.098606 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.099008 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.099009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.099009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.099144 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.099214 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.099607 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.099729 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:37:47.100095 26266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.088319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.66:45641)
0806 22:37:47.088503 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.088546 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.089113 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.089374 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.089377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.090003 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.090214 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.090220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.091213 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.091215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.092469 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.092473 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.092892 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.092893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.093064 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.093726 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.093739 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.095846 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.098038 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.098048 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.098052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.098444 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.098700 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.098701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.098701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.098905 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.099229 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.099237 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.099446 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.099586 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.099793 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260806 22:37:47.100689 26264 heartbeater.cc:347] Connected to a master server at 127.16.235.126:44375
I20260806 22:37:47.100839 26264 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:47.101054 26264 heartbeater.cc:511] Master 127.16.235.126:44375 requested a full tablet report, sending...
I20260806 22:37:47.101577 25810 ts_manager.cc:194] Registered new tserver with Master: 83c6bb1251604c9dad9f23e814657bcf (127.16.235.66:42963)
I20260806 22:37:47.101984 25810 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:45641
WARNING: no policy specified for oryx/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab.
Entry for principal oryx/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab.
I20260806 22:37:47.121739 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:44375
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab
--principal=oryx/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:47.208833 26270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:47.209088 26270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:47.209116 26270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:47.210295 26270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:47.210352 26270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:47.210383 26270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:47.212054 26270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab
--principal=oryx/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:44375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:47.212293 26270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:47.212496 26270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:47.218075 26279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.217967 26280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.217973 26283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:47.218415 26270 server_base.cc:1034] running on GCE node
I20260806 22:37:47.218724 26270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:47.219004 26270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:47.220182 26270 hybrid_clock.cc:648] HybridClock initialized: now 1786055867220172 us; error 44 us; skew 500 ppm
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:47.223088 26270 init.cc:377] Logged in from keytab as oryx/127.16.235.67@KRBTEST.COM (short username oryx)
I20260806 22:37:47.223579 26270 webserver.cc:460] Webserver started at http://127.16.235.67:39399/ using document root <none> and password file <none>
I20260806 22:37:47.223855 26270 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:47.223908 26270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:47.224098 26270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:47.225219 26270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "5f8eacbb46623c0e471188c6db1aa820"
server_key_iv: "48fda6f4bfb5a3f4b3add8a593e26618"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.225622 26270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "5f8eacbb46623c0e471188c6db1aa820"
server_key_iv: "48fda6f4bfb5a3f4b3add8a593e26618"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.227190 26270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:47.228060 26292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.228240 26270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:47.228353 26270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "5f8eacbb46623c0e471188c6db1aa820"
server_key_iv: "48fda6f4bfb5a3f4b3add8a593e26618"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.228449 26270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:47.243196 26270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:47.245384 26270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:47.245592 26270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:47.245864 26270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:47.246212 26270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:47.246270 26270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.246320 26270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:47.246348 26270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.252883 26270 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:44843
I20260806 22:37:47.252938 26452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:44843 every 8 connection(s)
I20260806 22:37:47.253319 26270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:47.258526 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 26270
I20260806 22:37:47.258632 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:47.258862 17325 external_mini_cluster.cc:1428] Setting key 75a486916c4816246d3ba2ecf130820a
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:47.263705 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.254157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:35561 (local address 127.16.235.126:44375)
0806 22:37:47.254224 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.254226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.254439 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:37:47.255062 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.255063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.255074 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.255137 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.256145 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.256653 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.257696 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.257884 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.257936 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.260714 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.260715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.260715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.260723 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.262344 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.262794 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.262795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.262795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.262851 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.263097 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.263098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.263098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.263211 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.263260 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.263569 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.263637 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:37:47.264004 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.254118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.67:35561)
0806 22:37:47.254297 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.254343 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.254966 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.255196 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.255199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.255839 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.256055 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.256061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.256771 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.256772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.257618 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.257620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.257958 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.257959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.258062 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.258801 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.258810 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.260629 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.262420 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.262427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.262429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.262714 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.262921 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.262922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.262922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.263031 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.263266 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.263271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.263438 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.263547 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.263710 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260806 22:37:47.264665 26454 heartbeater.cc:347] Connected to a master server at 127.16.235.126:44375
I20260806 22:37:47.264807 26454 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:47.265033 26454 heartbeater.cc:511] Master 127.16.235.126:44375 requested a full tablet report, sending...
I20260806 22:37:47.265439 25810 ts_manager.cc:194] Registered new tserver with Master: 0e2e71ea1beb4efe8dff23d039ff0b5a (127.16.235.67:44843)
I20260806 22:37:47.265856 25810 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:35561
I20260806 22:37:47.271168 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.235.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.125.keytab.
Entry for principal oryx/127.16.235.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.125.keytab.
I20260806 22:37:47.301981 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.125:43777
--webserver_interface=127.16.235.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.125.keytab
--principal=oryx/127.16.235.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:44375,127.16.235.125:43777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:47.399899 26459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:47.400151 26459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:47.400187 26459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:47.401520 26459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:47.401579 26459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:47.401598 26459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:47.401657 26459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:47.401678 26459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:47.403499 26459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:44375,127.16.235.125:43777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.125.keytab
--principal=oryx/127.16.235.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.125:43777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.235.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
W20260806 22:37:47.403666 26459 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.16.235.126:44375,127.16.235.125:43777'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 22:37:47.403852 26459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:47.404137 26459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:47.409772 26474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.409772 26470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.409812 26464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:47.410068 26459 server_base.cc:1034] running on GCE node
I20260806 22:37:47.410326 26459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:47.410633 26459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:47.411823 26459 hybrid_clock.cc:648] HybridClock initialized: now 1786055867411799 us; error 42 us; skew 500 ppm
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:47.414588 26459 init.cc:377] Logged in from keytab as oryx/127.16.235.125@KRBTEST.COM (short username oryx)
I20260806 22:37:47.415134 26459 webserver.cc:460] Webserver started at http://127.16.235.125:46825/ using document root <none> and password file <none>
I20260806 22:37:47.415401 26459 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:47.415452 26459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:47.415609 26459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:47.416759 26459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data/instance:
uuid: "85b2637944a540d3836f1591adba7a10"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "c65ef969c72e04d03fcf5d656e9b87a9"
server_key_iv: "fad399e07d3333270734f69d65343900"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.417163 26459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal/instance:
uuid: "85b2637944a540d3836f1591adba7a10"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "c65ef969c72e04d03fcf5d656e9b87a9"
server_key_iv: "fad399e07d3333270734f69d65343900"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.418781 26459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:37:47.419656 26484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.419906 26459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:47.419996 26459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal
uuid: "85b2637944a540d3836f1591adba7a10"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "c65ef969c72e04d03fcf5d656e9b87a9"
server_key_iv: "fad399e07d3333270734f69d65343900"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.420074 26459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:47.441668 26459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:47.444355 26459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:47.444550 26459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:47.448944 26459 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.125:43777
I20260806 22:37:47.448982 26556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.125:43777 every 8 connection(s)
I20260806 22:37:47.449400 26459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
I20260806 22:37:47.450457 26557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:47.452240 26557 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.125@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:47.459228 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 26459
I20260806 22:37:47.462371 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.453080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.16.235.126:44375)
0806 22:37:47.453150 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.453153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.453369 (+ 216us) server_negotiation.cc:407] Connection header received
0806 22:37:47.453913 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.453914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.453928 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.453995 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.454701 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.455245 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.456315 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.456517 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.456611 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.458930 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.458932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.458932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.458941 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.460658 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.461106 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.461107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.461108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.461177 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.461497 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.461498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.461499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.461683 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.461771 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.462179 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.462275 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:37:47.462598 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.453033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:44498)
0806 22:37:47.453230 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.453272 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.453789 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.454070 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.454075 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.454362 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.454579 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.454583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.455352 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.455353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.456214 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.456217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.456636 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.456637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.456770 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.457302 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.457311 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.458818 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.460757 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.460765 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.460767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.461017 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.461253 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.461254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.461255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.461424 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.461761 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.461767 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.461982 (+ 215us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.462148 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.462336 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260806 22:37:47.463518 26557 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.235.125@KRBTEST.COM
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.125@KRBTEST.COM for oryx/127.16.235.125@KRBTEST.COM
I20260806 22:37:47.469601 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.459871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.16.235.125:43777)
0806 22:37:47.460066 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.460072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.460133 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:37:47.460199 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.460202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.460216 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.460310 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.460976 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.461771 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.462575 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.463116 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.463181 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.466319 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.466326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.466327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.466333 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.468182 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.468582 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.468585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.468586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.468649 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.468876 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.468877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.468877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.469082 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.469282 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.469357 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.469455 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":107,"threads_started":1}
I20260806 22:37:47.471592 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.463835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.0.0.1:38510)
0806 22:37:47.463885 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.463897 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.463960 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.464264 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.464265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.464429 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.464496 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.464498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.465166 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.465167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.465631 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.465632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.466687 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.466688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.466752 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.467104 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.467108 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.468321 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.470775 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.470775 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.470776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.470938 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.471100 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.471100 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.471101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.471151 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.471359 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.471359 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.471399 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.471457 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.471532 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:37:47.471730 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.463894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.16.235.125:43777)
0806 22:37:47.464094 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.464096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.464102 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:37:47.464143 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.464143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.464186 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.464254 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.464586 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.465056 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.466325 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.466546 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.466725 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.468415 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.468417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.468417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.468422 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.470691 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.470995 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.470996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.470996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.471046 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.471196 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.471196 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.471196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.471299 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.471344 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.471475 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.471579 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":162,"thread_start_us":132,"threads_started":1}
I20260806 22:37:47.473009 26557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Bootstrap starting.
I20260806 22:37:47.473657 26557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:47.473942 26557 log.cc:824] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:47.474826 26557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: No bootstrap required, opened a new log
I20260806 22:37:47.476539 26557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } }
I20260806 22:37:47.476694 26557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:47.476744 26557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85b2637944a540d3836f1591adba7a10, State: Initialized, Role: FOLLOWER
I20260806 22:37:47.476882 26557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } }
I20260806 22:37:47.477212 26557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:47.477195 26565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } } }
I20260806 22:37:47.477315 26565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:37:47.479749 26585 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:37:47.479844 26585 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:37:47.481808 26459 master_runner.cc:428] Detected that this master 85b2637944a540d3836f1591adba7a10 is joining an existing cluster
I20260806 22:37:47.481876 26459 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.235.125:43777
I20260806 22:37:47.490248 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.483068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44506 (local address 127.16.235.126:44375)
0806 22:37:47.483138 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.483141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.483358 (+ 217us) server_negotiation.cc:407] Connection header received
0806 22:37:47.483391 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.483393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.483406 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.483477 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.483905 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.484406 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.485499 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.485684 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.485739 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.486954 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.486955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.486956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.486964 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.488925 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.489416 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.489418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.489418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.489492 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.489698 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.489699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.489700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.489831 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.489913 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.490064 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.490148 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:37:47.490262 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.483019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:44506)
0806 22:37:47.483222 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.483235 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.483309 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.483491 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.483493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.483732 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.483821 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.483824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.484489 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.484490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.485407 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.485411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.485773 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.485774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.485840 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.486337 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.486342 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.486858 (+ 516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.489042 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.489044 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.489045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.489334 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.489561 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.489561 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.489562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.489627 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.489902 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.489903 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.489948 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.490037 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.490148 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":126,"threads_started":1}
I20260806 22:37:47.493696 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.483408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.0.0.1:38526)
0806 22:37:47.486770 (+ 3362us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.486781 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.486883 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.487069 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.487070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.487283 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.487356 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.487358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.487991 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.487991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.489227 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.489229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.489756 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.489757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.489812 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.490246 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.490252 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.490740 (+ 488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.492574 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.492575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.492576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.492840 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.493053 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.493054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.493054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.493108 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.493362 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.493363 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.493386 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.493453 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.493563 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3336,"thread_start_us":89,"threads_started":1}
I20260806 22:37:47.493700 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.483354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.16.235.125:43777)
0806 22:37:47.483393 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.483395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.486867 (+ 3472us) server_negotiation.cc:407] Connection header received
0806 22:37:47.486979 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.486980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.486998 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.487051 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.487436 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.487909 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.489322 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.489636 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.489708 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.490832 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.490833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.490833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.490839 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.492500 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.492933 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.492934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.492935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.492993 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.493179 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.493180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.493181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.493298 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.493332 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.493490 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.493617 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":23}
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.126@KRBTEST.COM for oryx/127.16.235.125@KRBTEST.COM
I20260806 22:37:47.501375 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.492712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.0.0.1:38530)
0806 22:37:47.492872 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.492924 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.493003 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.493183 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.493186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.493545 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.493609 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.493611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.494551 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.494552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.495563 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.495567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.495938 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.495939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.496025 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.496651 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.496657 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.498318 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.500130 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.500140 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.500142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.500378 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.500613 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.500613 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.500614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.500683 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.500962 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.500970 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.501028 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.501129 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.501271 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260806 22:37:47.501368 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.492764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38530 (local address 127.16.235.125:43777)
0806 22:37:47.492803 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.492805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.492978 (+ 173us) server_negotiation.cc:407] Connection header received
0806 22:37:47.493108 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.493109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.493121 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.493167 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.493710 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.494449 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.495646 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.495844 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.495941 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.498420 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.498422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.498422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.498429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.500005 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.500471 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.500472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.500473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.500534 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.500769 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.500770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.500770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.500889 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.500970 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.501171 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.501265 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:37:47.502210 25810 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.235.125:43777: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "85b2637944a540d3836f1591adba7a10"
member_type: NON_VOTER
last_known_addr {
host: "127.16.235.125"
port: 43777
}
attrs {
promote: true
}
}
dest_uuid: "047a87bf14f8429686f8d19bb1ec9fb6"
cas_config_opid_index: -1
I20260806 22:37:47.502590 25810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } }
I20260806 22:37:47.503176 26596 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 85b2637944a540d3836f1591adba7a10 (127.16.235.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } } }
I20260806 22:37:47.503535 26526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:47.504302 26596 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.235.125:43777
I20260806 22:37:47.504493 26526 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Refusing update from remote peer 047a87bf14f8429686f8d19bb1ec9fb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 22:37:47.504531 26595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } } }
I20260806 22:37:47.504639 26595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:47.504691 26459 master.cc:537] Master@127.16.235.125:43777 shutting down...
I20260806 22:37:47.504952 26595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:37:47.505731 25793 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 22:37:47.506234 25793 consensus_peers.cc:487] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 -> Peer 85b2637944a540d3836f1591adba7a10 (127.16.235.125:43777): Couldn't send request to peer 85b2637944a540d3836f1591adba7a10. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:37:47.507894 26459 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:47.507993 26459 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:47.508046 26459 tablet_replica.cc:330] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: stopping tablet replica
I20260806 22:37:47.511034 26459 master.cc:552] Master@127.16.235.125:43777 shutdown complete.
I20260806 22:37:47.511098 26459 master_runner.cc:305] Clearing existing system tablet
I20260806 22:37:47.511246 26459 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:47.511315 26459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:47.511979 26459 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:47.512499 26601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.512691 26459 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:47.512756 26459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:47.512789 26459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal
uuid: "85b2637944a540d3836f1591adba7a10"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "c65ef969c72e04d03fcf5d656e9b87a9"
server_key_iv: "fad399e07d3333270734f69d65343900"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.513515 26459 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:47.514992 26459 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:37:47.515084 26459 log.cc:1196] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 22:37:47.515301 26459 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Deleting consensus metadata
I20260806 22:37:47.515451 26459 master_runner.cc:315] Copying system tablet from 127.16.235.126:44375
I20260806 22:37:47.516113 26459 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: tablet copy: Beginning tablet copy session from remote peer at address 127.16.235.126:44375
I20260806 22:37:47.523187 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.516633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44518 (local address 127.16.235.126:44375)
0806 22:37:47.516692 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.516694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.516790 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:37:47.516876 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.516877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.516889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.516949 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.517286 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.517822 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.518702 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.518944 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.519035 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.520135 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.520137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.520138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.520146 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.521759 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.522229 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.522231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.522231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.522303 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.522543 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.522544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.522544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.522753 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.522819 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.523041 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.523120 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:37:47.523211 26610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.516551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:44518)
0806 22:37:47.516689 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.516712 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.516809 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.516954 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.516955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.517142 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.517197 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.517199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.517924 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.517926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.518583 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.518586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.519037 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.519038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.519117 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.519681 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.519686 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.520021 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.521860 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.521862 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.521863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.522116 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.522401 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.522402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.522402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.522475 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.522810 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.522812 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.522875 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.522956 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.523094 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260806 22:37:47.523666 25839 tablet_copy_service.cc:140] P 047a87bf14f8429686f8d19bb1ec9fb6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 85b2637944a540d3836f1591adba7a10 ({username='oryx', principal='oryx/127.16.235.125@KRBTEST.COM'} at 127.0.0.1:44518)
I20260806 22:37:47.523828 25839 tablet_copy_service.cc:161] P 047a87bf14f8429686f8d19bb1ec9fb6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 85b2637944a540d3836f1591adba7a10 at {username='oryx', principal='oryx/127.16.235.125@KRBTEST.COM'} at 127.0.0.1:44518: session id = 85b2637944a540d3836f1591adba7a10-00000000000000000000000000000000
I20260806 22:37:47.524614 25839 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:37:47.525290 26459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:37:47.526419 26459 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: tablet copy: Starting download of 0 data blocks...
I20260806 22:37:47.526585 26459 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: tablet copy: Starting download of 1 WAL segments...
I20260806 22:37:47.527680 26459 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:37:47.528555 25839 tablet_copy_service.cc:342] P 047a87bf14f8429686f8d19bb1ec9fb6: Request end of tablet copy session 85b2637944a540d3836f1591adba7a10-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.235.125@KRBTEST.COM'} at 127.0.0.1:44518
I20260806 22:37:47.528604 25839 tablet_copy_service.cc:433] P 047a87bf14f8429686f8d19bb1ec9fb6: ending tablet copy session 85b2637944a540d3836f1591adba7a10-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 85b2637944a540d3836f1591adba7a10
I20260806 22:37:47.531472 26459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:47.532696 26611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.532765 26613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.532788 26618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:47.532989 26459 server_base.cc:1034] running on GCE node
I20260806 22:37:47.533146 26459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:47.533340 26459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:47.534504 26459 hybrid_clock.cc:648] HybridClock initialized: now 1786055867534490 us; error 33 us; skew 500 ppm
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:47.536010 26459 init.cc:377] Logged in from keytab as oryx/127.16.235.125@KRBTEST.COM (short username oryx)
I20260806 22:37:47.536338 26459 webserver.cc:460] Webserver started at http://127.16.235.125:33073/ using document root <none> and password file <none>
I20260806 22:37:47.536490 26459 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:47.536531 26459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:47.537197 26459 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:47.537809 26626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.537931 26459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:47.537981 26459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal
uuid: "85b2637944a540d3836f1591adba7a10"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "c65ef969c72e04d03fcf5d656e9b87a9"
server_key_iv: "fad399e07d3333270734f69d65343900"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.538049 26459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:47.563608 26459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:47.565567 26459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:47.569350 26459 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.125:43777
I20260806 22:37:47.569459 26697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.125:43777 every 8 connection(s)
I20260806 22:37:47.569748 26459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
I20260806 22:37:47.570887 26698 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:37:47.571470 26698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Bootstrap starting.
I20260806 22:37:47.575313 26698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:37:47.575585 26698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Bootstrap complete.
I20260806 22:37:47.575831 26698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } }
I20260806 22:37:47.575930 26698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 85b2637944a540d3836f1591adba7a10, State: Initialized, Role: LEARNER
I20260806 22:37:47.575973 26698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } }
I20260806 22:37:47.576205 26698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:47.576234 26703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } } }
I20260806 22:37:47.576375 26703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:37:47.579506 26718 catalog_manager.cc:1259] Loaded cluster ID: dbff21308e2f421bbbd3035d967e73e1
I20260806 22:37:47.579573 26718 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: loading cluster ID for follower catalog manager: success
I20260806 22:37:47.580298 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.571851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.16.235.125:43777)
0806 22:37:47.571977 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.571979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.571987 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:47.572011 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.572012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.572024 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.572105 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.572674 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.573494 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.574207 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.574433 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.574515 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.576127 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.576130 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.576130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.576142 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.577906 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.578417 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.578419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.578419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.578576 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.579352 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.579353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.579354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.579506 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.580045 (+ 539us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.580087 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.580173 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260806 22:37:47.580706 26718 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: acquiring CA information for follower catalog manager: success
I20260806 22:37:47.581120 26718 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
WARNING: no policy specified for oryx/127.16.235.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.124.keytab.
Entry for principal oryx/127.16.235.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.124.keytab.
I20260806 22:37:47.610229 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.124:39341
--webserver_interface=127.16.235.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.124.keytab
--principal=oryx/127.16.235.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:44375,127.16.235.125:43777,127.16.235.124:39341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:47.700101 26726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:47.700359 26726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:47.700384 26726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:47.701598 26726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:47.701644 26726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:47.701659 26726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:47.701673 26726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:47.701684 26726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:47.703244 26726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:44375,127.16.235.125:43777,127.16.235.124:39341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.124.keytab
--principal=oryx/127.16.235.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.124:39341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.235.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:47.703469 26726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:47.703661 26726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:47.708971 26733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.708982 26735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.708982 26732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:47.709326 26726 server_base.cc:1034] running on GCE node
I20260806 22:37:47.709604 26726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:47.709934 26726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:47.711112 26726 hybrid_clock.cc:648] HybridClock initialized: now 1786055867711097 us; error 50 us; skew 500 ppm
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:47.713857 26726 init.cc:377] Logged in from keytab as oryx/127.16.235.124@KRBTEST.COM (short username oryx)
I20260806 22:37:47.714363 26726 webserver.cc:460] Webserver started at http://127.16.235.124:43151/ using document root <none> and password file <none>
I20260806 22:37:47.714615 26726 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:47.714723 26726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:47.714890 26726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:47.716069 26726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data/instance:
uuid: "7eeb5d15afe14d7e9b257d568bd47440"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "04f4e974b2c0c3c5fcbebacd10ce2601"
server_key_iv: "2bbf1617c82c56eb57b005746a398f89"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.716487 26726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal/instance:
uuid: "7eeb5d15afe14d7e9b257d568bd47440"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "04f4e974b2c0c3c5fcbebacd10ce2601"
server_key_iv: "2bbf1617c82c56eb57b005746a398f89"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.717970 26726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:37:47.719046 26746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.719237 26726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:47.719364 26726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal
uuid: "7eeb5d15afe14d7e9b257d568bd47440"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "04f4e974b2c0c3c5fcbebacd10ce2601"
server_key_iv: "2bbf1617c82c56eb57b005746a398f89"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.719466 26726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:47.729506 26726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:47.731829 26726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:47.732028 26726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:47.737926 26726 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.124:39341
I20260806 22:37:47.738140 26830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.124:39341 every 8 connection(s)
I20260806 22:37:47.738484 26726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
I20260806 22:37:47.739775 26833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:47.741634 26833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:37:47.747301 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 26726
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.124@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:47.753355 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.742619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44534 (local address 127.16.235.126:44375)
0806 22:37:47.742731 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.742734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.742806 (+ 72us) server_negotiation.cc:407] Connection header received
0806 22:37:47.743318 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.743319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.743331 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.743399 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.744043 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.744609 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.746236 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.746529 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.746603 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.749760 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.749775 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.749776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.749789 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.751644 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.752231 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.752232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.752233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.752329 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.752607 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.752608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.752608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.752786 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.752872 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.753144 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.753243 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 22:37:47.753414 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.742389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:44534)
0806 22:37:47.742643 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.742717 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.743214 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.743456 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.743459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.743734 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.743953 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.743957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.744743 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.744744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.746092 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.746096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.746685 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.746687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.746859 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.747545 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.747557 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.749617 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.751769 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.751778 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.751781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.752117 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.752418 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.752419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.752420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.752535 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.752866 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.752871 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.753000 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.753089 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.753219 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":115,"threads_started":1}
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.235.124@KRBTEST.COM
I20260806 22:37:47.754099 26833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:37:47.757735 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.747966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.16.235.124:39341)
0806 22:37:47.748383 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.748388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.748416 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:37:47.748440 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.748442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.748465 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.748587 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.749457 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.750237 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.751294 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.751610 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.751731 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.754757 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.754765 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.754765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.754772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.756426 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.756815 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.756819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.756819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.756883 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.757102 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.757103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.757103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.757280 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.757444 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.757517 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.757602 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":111,"threads_started":1}
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.124@KRBTEST.COM for oryx/127.16.235.125@KRBTEST.COM
I20260806 22:37:47.760725 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.754423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.0.0.1:38564)
0806 22:37:47.754486 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.754498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.754546 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.754743 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.754744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.754917 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.754972 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.754973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.755666 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.755667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.756184 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.756185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.756602 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.756603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.756684 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.757100 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.757105 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.758490 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.759905 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.759906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.759906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.760090 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.760252 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.760253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.760253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.760309 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.760499 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.760499 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.760534 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.760585 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.760659 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:37:47.760728 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.754493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.16.235.125:43777)
0806 22:37:47.754537 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.754538 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.754547 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:47.754606 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.754606 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.754618 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.754729 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.755059 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.755557 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.756305 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.756523 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.756606 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.758591 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.758593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.758594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.758601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.759832 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.760148 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.760149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.760149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.760213 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.760347 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.760348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.760348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.760445 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.760491 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.760600 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.760667 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:37:47.761317 26833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.124@KRBTEST.COM for oryx/127.16.235.124@KRBTEST.COM
I20260806 22:37:47.767570 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.761590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.0.0.1:52892)
0806 22:37:47.761641 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.761652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.761703 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.761819 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.761820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.761947 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.761981 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.761982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.762581 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.762581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.763179 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.763181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.763552 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.763553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.763613 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.763981 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.763986 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.765174 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.766572 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.766573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.766573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.766827 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.767061 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.767061 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.767062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.767117 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.767309 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.767309 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.767345 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.767397 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.767481 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:37:47.767621 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.761663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52892 (local address 127.16.235.124:39341)
0806 22:37:47.761717 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.761719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.761744 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:37:47.761768 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.761769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.761779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.761815 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.762040 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.762507 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.763251 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.763450 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.763555 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.765255 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.765256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.765257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.765263 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.766497 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.766897 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.766897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.766898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.766994 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.767157 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.767158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.767158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.767256 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.767302 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.767414 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.767526 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":36}
I20260806 22:37:47.769219 26833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: Bootstrap starting.
I20260806 22:37:47.770083 26833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:47.770467 26833 log.cc:824] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:47.771322 26833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: No bootstrap required, opened a new log
I20260806 22:37:47.772940 26833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } }
I20260806 22:37:47.773077 26833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:47.773101 26833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eeb5d15afe14d7e9b257d568bd47440, State: Initialized, Role: FOLLOWER
I20260806 22:37:47.773247 26833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } }
I20260806 22:37:47.773542 26839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } } }
I20260806 22:37:47.773617 26833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:47.773635 26839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:37:47.775676 26862 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:37:47.775748 26862 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:37:47.778429 26726 master_runner.cc:428] Detected that this master 7eeb5d15afe14d7e9b257d568bd47440 is joining an existing cluster
I20260806 22:37:47.778508 26726 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.235.124:39341
I20260806 22:37:47.788431 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.780917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.0.0.1:52900)
0806 22:37:47.781063 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.781072 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.781134 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.781282 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.781282 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.781590 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.781658 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.781659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.782538 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.782539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.783556 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.783559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.784093 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.784094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.784155 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.785105 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.785110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.785507 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.787379 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.787380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.787380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.787574 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.787800 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.787801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.787801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.787871 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.788156 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.788156 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.788194 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.788246 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.788343 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":55,"thread_start_us":58,"threads_started":1}
I20260806 22:37:47.788533 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.780840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.16.235.124:39341)
0806 22:37:47.780882 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.780883 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.781115 (+ 232us) server_negotiation.cc:407] Connection header received
0806 22:37:47.781194 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.781194 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.781203 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.781229 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.781735 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.782443 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.783661 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.783949 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.783993 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.785614 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.785616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.785616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.785624 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.787280 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.787654 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.787656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.787656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.787726 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.787938 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.787939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.787939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.788087 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.788162 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.788270 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.788416 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":22}
I20260806 22:37:47.809505 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 25904
I20260806 22:37:47.811702 26869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.780504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.0.0.1:38568)
0806 22:37:47.780716 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.780726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.780811 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.780994 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.780995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.781396 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.781473 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.781476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.782500 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.782501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.783659 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.783662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.784189 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.784190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.784270 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.807423 (+ 23153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.807430 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.808109 (+ 679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.810437 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.810438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.810439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.810710 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.810936 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.810937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.810937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.811004 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.811302 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.811303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.811353 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.811431 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.811558 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":43,"thread_start_us":90,"threads_started":1}
I20260806 22:37:47.812116 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.780500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.16.235.125:43777)
0806 22:37:47.780555 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.780558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.780786 (+ 228us) server_negotiation.cc:407] Connection header received
0806 22:37:47.780889 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.780889 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.780900 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.780930 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.781579 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.782394 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.783766 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.784063 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.784129 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.808287 (+ 24158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.808289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.808289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.808298 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.810309 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.810800 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.810801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.810801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.810864 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.811070 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.811070 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.811071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.811217 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.811277 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.811931 (+ 654us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.812025 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:37:47.822219 26870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.780434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:44540)
0806 22:37:47.780653 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.780667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.780759 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.780983 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.780984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.781359 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.781552 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.781555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.782551 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.782551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.783556 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.783559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.784097 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.784097 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.784155 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.807482 (+ 23327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.807488 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.808086 (+ 598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.820697 (+ 12611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.820699 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.820700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.821000 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.821286 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.821287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.821287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.821347 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.821814 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.821815 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.821875 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.821954 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.822089 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":34,"thread_start_us":98,"threads_started":1}
I20260806 22:37:47.822286 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.780605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44540 (local address 127.16.235.126:44375)
0806 22:37:47.780669 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.780671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.780728 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:37:47.780846 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.780848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.780860 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.780893 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.781664 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.782457 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.783670 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.783970 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.784009 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.808271 (+ 24262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.808273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.808274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.808284 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.819091 (+ 10807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.821124 (+ 2033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.821126 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.821127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.821207 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.821478 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.821480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.821480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.821686 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.821775 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.822008 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.822118 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:37:47.825970 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:42679
--local_ip_for_outbound_sockets=127.16.235.65
--tserver_master_addrs=127.16.235.126:44375
--webserver_port=41347
--webserver_interface=127.16.235.65
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab
--principal=oryx/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.235.126:44375,127.16.235.125:43777
--tserver_master_addrs=127.16.235.126:44375,127.16.235.125:43777,127.16.235.124:39341 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055866, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.126@KRBTEST.COM for oryx/127.16.235.124@KRBTEST.COM
I20260806 22:37:47.837037 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.828049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.0.0.1:52902)
0806 22:37:47.828108 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.828119 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.828160 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.828380 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.828382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.828613 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.828677 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.828679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.829635 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.829636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.830562 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.830566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.831086 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.831088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.831177 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.831739 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.831746 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.833522 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.835724 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.835725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.835726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.835992 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.836250 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.836251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.836252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.836323 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.836661 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.836662 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.836716 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.836793 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.836927 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:37:47.837077 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.828111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52902 (local address 127.16.235.124:39341)
0806 22:37:47.828239 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.828241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.828249 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:47.828285 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.828286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.828297 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.828351 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.828773 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.829534 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.830686 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.830994 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.831072 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.833632 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.833635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.833635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.833645 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.835609 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.836083 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.836084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.836085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.836158 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.836402 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.836404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.836404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.836580 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.836664 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.836843 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.836940 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 22:37:47.838289 25810 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.235.124:39341: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440"
member_type: NON_VOTER
last_known_addr {
host: "127.16.235.124"
port: 39341
}
attrs {
promote: true
}
}
dest_uuid: "047a87bf14f8429686f8d19bb1ec9fb6"
cas_config_opid_index: 5
I20260806 22:37:47.838626 25810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } }
I20260806 22:37:47.839242 26596 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7eeb5d15afe14d7e9b257d568bd47440 (127.16.235.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.839664 26782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:47.840344 26596 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.235.124:39341
I20260806 22:37:47.840422 26596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.840481 26596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:47.840695 26782 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 FOLLOWER]: Refusing update from remote peer 047a87bf14f8429686f8d19bb1ec9fb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:37:47.840791 26726 master.cc:537] Master@127.16.235.124:39341 shutting down...
I20260806 22:37:47.841147 26595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:37:47.843999 25793 consensus_peers.cc:487] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 -> Peer 7eeb5d15afe14d7e9b257d568bd47440 (127.16.235.124:39341): Couldn't send request to peer 7eeb5d15afe14d7e9b257d568bd47440. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:37:47.845006 26726 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:47.845112 26726 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:47.845171 26726 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: stopping tablet replica
I20260806 22:37:47.846279 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.839252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.0.0.1:38584)
0806 22:37:47.841591 (+ 2339us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.841605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.841661 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.841917 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.841918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.841919 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:47.841971 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.841973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.842992 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.842993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.844389 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.844393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.845992 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.845994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.846069 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.846098 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.846148 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.846198 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2319}
I20260806 22:37:47.846624 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.839346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.16.235.125:43777)
0806 22:37:47.841489 (+ 2143us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.841491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.841719 (+ 228us) server_negotiation.cc:407] Connection header received
0806 22:37:47.841757 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.841758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.841786 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.841845 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:47.842056 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.842882 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.844503 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.845793 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.845873 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.845954 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.846512 (+ 558us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.846559 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2110}
I20260806 22:37:47.847343 26664 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 LEARNER]: Refusing update from remote peer 047a87bf14f8429686f8d19bb1ec9fb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:37:47.847563 26595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 22:37:47.848206 26664 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7eeb5d15afe14d7e9b257d568bd47440 (127.16.235.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.849545 26875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.849684 26875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:37:47.849658 26703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 047a87bf14f8429686f8d19bb1ec9fb6. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: NON_VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: true } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.849920 26703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:37:47.850003 26595 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: attempting to promote NON_VOTER 85b2637944a540d3836f1591adba7a10 to VOTER
I20260806 22:37:47.850226 26595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } }
I20260806 22:37:47.850811 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.850687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.0.0.1:52910)
0806 22:37:47.850733 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.850779 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:39341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:37:47.851155 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.851033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.0.0.1:52920)
0806 22:37:47.851091 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.851127 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:39341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:37:47.851258 26664 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 LEARNER]: Refusing update from remote peer 047a87bf14f8429686f8d19bb1ec9fb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260806 22:37:47.851294 25793 consensus_peers.cc:487] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 -> Peer 7eeb5d15afe14d7e9b257d568bd47440 (127.16.235.124:39341): Couldn't send request to peer 7eeb5d15afe14d7e9b257d568bd47440. Status: Network error: Client connection negotiation failed: client connection to 127.16.235.124:39341: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:37:47.851465 26596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 22:37:47.852130 26877 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 85b2637944a540d3836f1591adba7a10 (127.16.235.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.852289 26664 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 85b2637944a540d3836f1591adba7a10 (127.16.235.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.853243 26703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.853330 26703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:47.853678 26596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.853760 26596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:47.853848 26877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.853904 26877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:47.858892 26726 master.cc:552] Master@127.16.235.124:39341 shutdown complete.
I20260806 22:37:47.858960 26726 master_runner.cc:305] Clearing existing system tablet
I20260806 22:37:47.859102 26726 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:47.859158 26726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:47.859980 26726 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:47.860613 26880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.860749 26726 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:47.860800 26726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:37:47.860821 26726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal
uuid: "7eeb5d15afe14d7e9b257d568bd47440"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "04f4e974b2c0c3c5fcbebacd10ce2601"
server_key_iv: "2bbf1617c82c56eb57b005746a398f89"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.861660 26726 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:47.863322 26726 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:37:47.863407 26726 log.cc:1196] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 22:37:47.863643 26726 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: Deleting consensus metadata
I20260806 22:37:47.863803 26726 master_runner.cc:315] Copying system tablet from 127.16.235.126:44375
I20260806 22:37:47.864477 26726 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: tablet copy: Beginning tablet copy session from remote peer at address 127.16.235.126:44375
I20260806 22:37:47.872608 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.864857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:44556)
0806 22:37:47.865052 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.865063 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.865133 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.865312 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.865313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.865535 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.865606 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.865608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.866580 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.866581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.867496 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.867506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.868007 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.868009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.868082 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.868686 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.868691 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.869133 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.871278 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.871280 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.871280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.871547 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.871827 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.871828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.871829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.871905 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.872226 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.872227 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.872284 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.872372 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.872498 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":76,"threads_started":1}
I20260806 22:37:47.872612 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.864908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.16.235.126:44375)
0806 22:37:47.865003 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.865006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.865145 (+ 139us) server_negotiation.cc:407] Connection header received
0806 22:37:47.865213 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.865214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.865227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.865290 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.865707 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.866471 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.867618 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.867908 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.867992 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.869244 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.869245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.869245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.869254 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.871175 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.871646 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.871647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.871647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.871736 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.871986 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.871987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.871988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.872146 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.872228 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.872426 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.872517 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:37:47.873342 25839 tablet_copy_service.cc:140] P 047a87bf14f8429686f8d19bb1ec9fb6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7eeb5d15afe14d7e9b257d568bd47440 ({username='oryx', principal='oryx/127.16.235.124@KRBTEST.COM'} at 127.0.0.1:44556)
I20260806 22:37:47.873411 25839 tablet_copy_service.cc:161] P 047a87bf14f8429686f8d19bb1ec9fb6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7eeb5d15afe14d7e9b257d568bd47440 at {username='oryx', principal='oryx/127.16.235.124@KRBTEST.COM'} at 127.0.0.1:44556: session id = 7eeb5d15afe14d7e9b257d568bd47440-00000000000000000000000000000000
I20260806 22:37:47.874080 25839 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:37:47.874794 26726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:37:47.876554 26726 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: tablet copy: Starting download of 0 data blocks...
I20260806 22:37:47.876744 26726 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: tablet copy: Starting download of 1 WAL segments...
I20260806 22:37:47.877732 26726 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:37:47.878866 25839 tablet_copy_service.cc:342] P 047a87bf14f8429686f8d19bb1ec9fb6: Request end of tablet copy session 7eeb5d15afe14d7e9b257d568bd47440-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.235.124@KRBTEST.COM'} at 127.0.0.1:44556
I20260806 22:37:47.878944 25839 tablet_copy_service.cc:433] P 047a87bf14f8429686f8d19bb1ec9fb6: ending tablet copy session 7eeb5d15afe14d7e9b257d568bd47440-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7eeb5d15afe14d7e9b257d568bd47440
I20260806 22:37:47.882232 26726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:47.883651 26893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.883875 26897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:47.887318 26726 server_base.cc:1034] running on GCE node
W20260806 22:37:47.887677 26895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:47.887898 26726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:47.888073 26726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:47.889231 26726 hybrid_clock.cc:648] HybridClock initialized: now 1786055867889219 us; error 35 us; skew 500 ppm
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:47.891014 26726 init.cc:377] Logged in from keytab as oryx/127.16.235.124@KRBTEST.COM (short username oryx)
I20260806 22:37:47.891345 26726 webserver.cc:460] Webserver started at http://127.16.235.124:44219/ using document root <none> and password file <none>
I20260806 22:37:47.891489 26726 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:47.891528 26726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:47.892431 26726 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:47.893085 26906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.893262 26726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.893323 26726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal
uuid: "7eeb5d15afe14d7e9b257d568bd47440"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "04f4e974b2c0c3c5fcbebacd10ce2601"
server_key_iv: "2bbf1617c82c56eb57b005746a398f89"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.893401 26726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:47.906282 26726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:47.908458 26726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:47.912338 26726 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.124:39341
I20260806 22:37:47.912475 26988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.124:39341 every 8 connection(s)
I20260806 22:37:47.912735 26726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
I20260806 22:37:47.913738 26989 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:37:47.914454 26989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: Bootstrap starting.
I20260806 22:37:47.918222 26989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:37:47.918527 26989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: Bootstrap complete.
I20260806 22:37:47.918995 26989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } }
I20260806 22:37:47.919107 26989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7eeb5d15afe14d7e9b257d568bd47440, State: Initialized, Role: LEARNER
I20260806 22:37:47.919150 26989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } }
I20260806 22:37:47.919414 26989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:47.919387 26993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:47.919643 26993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:37:47.921427 27008 catalog_manager.cc:1259] Loaded cluster ID: dbff21308e2f421bbbd3035d967e73e1
I20260806 22:37:47.921787 27008 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: loading cluster ID for follower catalog manager: success
I20260806 22:37:47.923004 27008 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: acquiring CA information for follower catalog manager: success
I20260806 22:37:47.923319 27008 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 22:37:47.938699 26873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:47.939054 26873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:47.939103 26873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:47.940389 26873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:47.940470 26873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:47.940519 26873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:47.942180 26873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab
--principal=oryx/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:42679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=41347
--tserver_master_addrs=127.16.235.126:44375,127.16.235.125:43777,127.16.235.124:39341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:47.942480 26873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:47.942827 26873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:47.948415 27015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.949096 27014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:47.949108 27017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:47.950542 26873 server_base.cc:1034] running on GCE node
I20260806 22:37:47.950825 26873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:47.951122 26873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:47.952301 26873 hybrid_clock.cc:648] HybridClock initialized: now 1786055867952289 us; error 47 us; skew 500 ppm
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:47.955179 26873 init.cc:377] Logged in from keytab as oryx/127.16.235.65@KRBTEST.COM (short username oryx)
I20260806 22:37:47.955762 26873 webserver.cc:460] Webserver started at http://127.16.235.65:41347/ using document root <none> and password file <none>
I20260806 22:37:47.956027 26873 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:47.956120 26873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:47.957634 26873 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:47.958280 27029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.958470 26873 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:47.958581 26873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "6bcee1edca234af1845705222b536dff"
format_stamp: "Formatted at 2026-08-06 22:37:46 on dist-test-slave-3kk6"
server_key: "1aa7865e9a9dc396f23f6945cc20608a"
server_key_iv: "4d1a9370ee62cb85649fc473dc9ffef7"
server_key_version: "encryptionkey@0"
I20260806 22:37:47.959064 26873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:47.968851 26873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:47.971534 26873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:47.971746 26873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:47.972059 26873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:47.972416 26873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:47.972473 26873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.972517 26873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:47.972560 26873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:47.981073 26873 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:42679
I20260806 22:37:47.981115 27187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:42679 every 8 connection(s)
I20260806 22:37:47.981532 26873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:47.985190 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 26873
I20260806 22:37:47.985353 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 26087
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.124@KRBTEST.COM
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.125@KRBTEST.COM
Aug 06 22:37:47 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055867, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:47.998219 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.982781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:53983 (local address 127.16.235.125:43777)
0806 22:37:47.982830 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.982832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.982879 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:37:47.983975 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.983977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.983991 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.984043 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.984849 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.985603 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.988212 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.988495 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.988547 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.992202 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.992204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.992204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.992216 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.993854 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.996757 (+ 2903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.996759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.996759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.996825 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.997220 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.997222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.997222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.997420 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.997480 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.997980 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.998085 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:37:47.998859 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.982326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:58397 (local address 127.16.235.126:44375)
0806 22:37:47.982414 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.982417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.982815 (+ 398us) server_negotiation.cc:407] Connection header received
0806 22:37:47.983978 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.983980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.983994 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.984032 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.984874 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.985640 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.988958 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.989277 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.989327 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.994130 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.994132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.994132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.994142 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.996519 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.997187 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.997188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.997189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.997257 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.997798 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.997799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.997799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.997965 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.998022 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.998628 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.998762 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:37:47.999331 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.982474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:54335 (local address 127.16.235.124:39341)
0806 22:37:47.982586 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:47.982588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:47.982742 (+ 154us) server_negotiation.cc:407] Connection header received
0806 22:37:47.983979 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:47.983981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:47.983996 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:47.984051 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:47.984866 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.985885 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.987448 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.987769 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.987814 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.994466 (+ 6652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:47.994468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:47.994468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:47.994478 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:47.996408 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.997478 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.997480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.997480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.997549 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:47.998193 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:47.998194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:47.998195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:47.998348 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:47.998412 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:47.999135 (+ 723us) server_negotiation.cc:299] Negotiation successful
0806 22:37:47.999237 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
I20260806 22:37:47.999663 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.982703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.16.235.65:53983)
0806 22:37:47.982815 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.982826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.983835 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.984121 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.984125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.984484 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.984746 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.984751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.985736 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.985737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.988109 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.988111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.988636 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.988637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.988719 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.989421 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.989426 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.992065 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.993953 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.993964 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.993967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.994297 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.996904 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.996905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.996905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.997089 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.997517 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.997526 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.997797 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.998720 (+ 923us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.999160 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":1443,"thread_start_us":70,"threads_started":1}
I20260806 22:37:47.999861 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.982343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.65:58397)
0806 22:37:47.982756 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.982766 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.983866 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.984123 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.984126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.984515 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.984747 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.984755 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.985736 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.985737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.987331 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.987334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.990044 (+ 2710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.990045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.990114 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.990611 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.990616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.993139 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.996669 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.996670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.996670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.996908 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.997333 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.997334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.997334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.997389 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.998057 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.998058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.998096 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.999001 (+ 905us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.999733 (+ 732us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":705,"thread_start_us":110,"threads_started":1}
I20260806 22:37:47.999663 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:47.982331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.16.235.65:54335)
0806 22:37:47.982527 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:47.982573 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:47.983872 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:47.984123 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:47.984126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:47.984524 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:47.984747 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.984755 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.986090 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.986091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.987331 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:47.987344 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:47.987910 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:47.987911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:47.988058 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:47.988736 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:47.988748 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:47.994012 (+ 5264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:47.996526 (+ 2514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.996527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.996528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.997089 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.997619 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:47.997619 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:47.997619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:47.997672 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:47.998457 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:47.998458 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:47.998499 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:37:47.999456 (+ 957us) client_negotiation.cc:240] Negotiation successful
0806 22:37:47.999565 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":38,"thread_start_us":112,"threads_started":1}
I20260806 22:37:48.000996 27190 heartbeater.cc:347] Connected to a master server at 127.16.235.125:43777
I20260806 22:37:48.001120 27190 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:48.001205 27188 heartbeater.cc:347] Connected to a master server at 127.16.235.124:39341
I20260806 22:37:48.001236 27188 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:48.001351 27190 heartbeater.cc:511] Master 127.16.235.125:43777 requested a full tablet report, sending...
I20260806 22:37:48.001351 27188 heartbeater.cc:511] Master 127.16.235.124:39341 requested a full tablet report, sending...
I20260806 22:37:48.001983 26647 ts_manager.cc:194] Registered new tserver with Master: 6bcee1edca234af1845705222b536dff (127.16.235.65:42679)
I20260806 22:37:48.001983 26934 ts_manager.cc:194] Registered new tserver with Master: 6bcee1edca234af1845705222b536dff (127.16.235.65:42679)
I20260806 22:37:48.002259 27189 heartbeater.cc:347] Connected to a master server at 127.16.235.126:44375
I20260806 22:37:48.002446 27189 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:48.002528 27189 heartbeater.cc:511] Master 127.16.235.126:44375 requested a full tablet report, sending...
I20260806 22:37:48.002859 25810 ts_manager.cc:194] Re-registered known tserver with Master: 6bcee1edca234af1845705222b536dff (127.16.235.65:42679)
I20260806 22:37:48.003276 25810 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:58397
I20260806 22:37:48.014525 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:42963
--local_ip_for_outbound_sockets=127.16.235.66
--tserver_master_addrs=127.16.235.126:44375
--webserver_port=36165
--webserver_interface=127.16.235.66
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab
--principal=oryx/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.235.126:44375,127.16.235.125:43777
--tserver_master_addrs=127.16.235.126:44375,127.16.235.125:43777,127.16.235.124:39341 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:48.109665 27201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:48.110021 27201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:48.110071 27201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:48.112454 27201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:48.112564 27201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:48.112620 27201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:48.115465 27201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab
--principal=oryx/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:42963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=36165
--tserver_master_addrs=127.16.235.126:44375,127.16.235.125:43777,127.16.235.124:39341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:48.115818 27201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:48.116134 27201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:48.123050 27207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:48.123157 27209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:48.123286 27201 server_base.cc:1034] running on GCE node
W20260806 22:37:48.123026 27206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:48.123939 27201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:48.124284 27201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:48.125487 27201 hybrid_clock.cc:648] HybridClock initialized: now 1786055868125464 us; error 42 us; skew 500 ppm
Aug 06 22:37:48 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055868, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:48.130505 27201 init.cc:377] Logged in from keytab as oryx/127.16.235.66@KRBTEST.COM (short username oryx)
I20260806 22:37:48.131103 27201 webserver.cc:460] Webserver started at http://127.16.235.66:36165/ using document root <none> and password file <none>
I20260806 22:37:48.131424 27201 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:48.131526 27201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:48.133116 27201 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:37:48.133801 27220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:48.134033 27201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:48.134147 27201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "83c6bb1251604c9dad9f23e814657bcf"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "b31bfc6d0241d1d307c7586f6b505716"
server_key_iv: "9bb1f1cc7aefe65fc53cee2ebbb7ad4f"
server_key_version: "encryptionkey@0"
I20260806 22:37:48.134544 27201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:48.158038 27201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:48.160709 27201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:48.160928 27201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:48.161316 27201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:48.161677 27201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:48.161712 27201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:48.161739 27201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:48.161753 27201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:48.168265 27201 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:42963
I20260806 22:37:48.168313 27379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:42963 every 8 connection(s)
I20260806 22:37:48.168712 27201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:48.171587 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 27201
I20260806 22:37:48.171792 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 26270
Aug 06 22:37:48 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055868, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
Aug 06 22:37:48 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055868, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.125@KRBTEST.COM
Aug 06 22:37:48 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055868, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.124@KRBTEST.COM
I20260806 22:37:48.184002 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.169629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:43119 (local address 127.16.235.126:44375)
0806 22:37:48.169760 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.169765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.169960 (+ 195us) server_negotiation.cc:407] Connection header received
0806 22:37:48.171488 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.171490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.171505 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.171541 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.172624 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.173287 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.174762 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.175087 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.175130 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.178315 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.178317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.178317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.178328 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.180262 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.181148 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.181150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.181150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.182330 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.182841 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.182843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.182843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.183056 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.183117 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.183692 (+ 575us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.183837 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 22:37:48.184197 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.170451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:54107 (local address 127.16.235.124:39341)
0806 22:37:48.170546 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.170548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.170559 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:48.171514 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.171516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.171534 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.171570 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.174612 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.175630 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.177048 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.177477 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.177529 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.180906 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.180908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.180909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.180917 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.182822 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.183265 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.183266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.183267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.183328 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.183549 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.183550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.183550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.183705 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.183750 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.183980 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.184093 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:37:48.184268 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.170042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:60655 (local address 127.16.235.125:43777)
0806 22:37:48.170118 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.170120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.170242 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:37:48.171489 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.171490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.171505 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.171541 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.174550 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.175307 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.176760 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.177051 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.177098 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.180978 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.180980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.180980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.180989 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.183041 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.183443 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.183445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.183445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.183503 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.183706 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.183707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.183707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.183843 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.183888 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.184089 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.184171 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:37:48.184897 27389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.169984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.16.235.66:60655)
0806 22:37:48.170162 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.170173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.171362 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.171791 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.171792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.174337 (+ 2545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:48.174432 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.174435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.175414 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.175415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.176638 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.176642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.177184 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.177185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.177266 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.177852 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:48.177858 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:48.180875 (+ 3017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:48.183129 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.183130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.183131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.183340 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.183582 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.183583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.183583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.183632 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.183937 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:48.183938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:48.183974 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.184406 (+ 432us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.184644 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":121,"threads_started":1}
I20260806 22:37:48.185226 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.169598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.66:43119)
0806 22:37:48.169789 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.169835 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.171345 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.171770 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.171774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.172240 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:48.172519 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.172525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.173387 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.173388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.174646 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.174670 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.175225 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.175226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.175377 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.176065 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:48.176076 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:48.178186 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:48.180366 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.180427 (+ 61us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.180432 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.181018 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.182448 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.182449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.182449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.182710 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.183181 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:48.183192 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:48.183453 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.184521 (+ 1068us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.184792 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":61,"thread_start_us":102,"threads_started":1}
I20260806 22:37:48.185391 27390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.170291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.16.235.66:54107)
0806 22:37:48.170455 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.170467 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.171376 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.171787 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.171788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.174390 (+ 2602us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:48.174477 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.174481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.175755 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.175756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.176923 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.176927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.177644 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.177646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.177786 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.178250 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:48.178256 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:48.180773 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:48.182941 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.182942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.182942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.183177 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.183408 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.183409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.183409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.183468 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.183801 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:48.183801 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:48.183857 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.184558 (+ 701us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.184894 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":215,"thread_start_us":89,"threads_started":1}
I20260806 22:37:48.185833 27384 heartbeater.cc:347] Connected to a master server at 127.16.235.125:43777
I20260806 22:37:48.185818 27381 heartbeater.cc:347] Connected to a master server at 127.16.235.126:44375
I20260806 22:37:48.185979 27381 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:48.185981 27384 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:48.186236 27381 heartbeater.cc:511] Master 127.16.235.126:44375 requested a full tablet report, sending...
I20260806 22:37:48.186236 27384 heartbeater.cc:511] Master 127.16.235.125:43777 requested a full tablet report, sending...
I20260806 22:37:48.186730 25810 ts_manager.cc:194] Re-registered known tserver with Master: 83c6bb1251604c9dad9f23e814657bcf (127.16.235.66:42963)
I20260806 22:37:48.187150 25810 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:43119
I20260806 22:37:48.187440 26647 ts_manager.cc:194] Registered new tserver with Master: 83c6bb1251604c9dad9f23e814657bcf (127.16.235.66:42963)
I20260806 22:37:48.200457 27380 heartbeater.cc:347] Connected to a master server at 127.16.235.124:39341
I20260806 22:37:48.200582 27380 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:48.200721 27380 heartbeater.cc:511] Master 127.16.235.124:39341 requested a full tablet report, sending...
I20260806 22:37:48.201246 26934 ts_manager.cc:194] Registered new tserver with Master: 83c6bb1251604c9dad9f23e814657bcf (127.16.235.66:42963)
I20260806 22:37:48.204597 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:44843
--local_ip_for_outbound_sockets=127.16.235.67
--tserver_master_addrs=127.16.235.126:44375
--webserver_port=39399
--webserver_interface=127.16.235.67
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab
--principal=oryx/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.235.126:44375,127.16.235.125:43777
--tserver_master_addrs=127.16.235.126:44375,127.16.235.125:43777,127.16.235.124:39341 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:48.240696 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.237434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58954 (local address 127.16.235.124:39341)
0806 22:37:48.237587 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.237592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.237602 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:37:48.237635 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.237636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.237684 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.237749 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:48.237961 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.238454 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.239929 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.240337 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.240416 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.240465 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.240582 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.240617 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 22:37:48.240702 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.237284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.0.0.1:58954)
0806 22:37:48.237357 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.237371 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.237449 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.237790 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.237792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.237795 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:48.237859 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.237861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.238579 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.238581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.239789 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.239793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.240436 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.240437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.240506 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.240510 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.240572 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.240621 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:37:48.241197 26946 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 22:37:48.241405 26993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 047a87bf14f8429686f8d19bb1ec9fb6. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:48.241528 26993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: This master's current role is: LEARNER
W20260806 22:37:48.292695 27391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:48.292977 27391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:48.293021 27391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:48.294286 27391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:48.294358 27391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:48.294404 27391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:48.296164 27391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab
--principal=oryx/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:44843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=39399
--tserver_master_addrs=127.16.235.126:44375,127.16.235.125:43777,127.16.235.124:39341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:48.296466 27391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:48.296818 27391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:48.302412 27399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:48.302412 27398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:48.302486 27391 server_base.cc:1034] running on GCE node
W20260806 22:37:48.302459 27401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:48.303042 27391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:48.303330 27391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:48.304482 27391 hybrid_clock.cc:648] HybridClock initialized: now 1786055868304459 us; error 40 us; skew 500 ppm
Aug 06 22:37:48 dist-test-slave-3kk6 krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055868, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:48.307148 27391 init.cc:377] Logged in from keytab as oryx/127.16.235.67@KRBTEST.COM (short username oryx)
I20260806 22:37:48.307682 27391 webserver.cc:460] Webserver started at http://127.16.235.67:39399/ using document root <none> and password file <none>
I20260806 22:37:48.307952 27391 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:48.308024 27391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:48.309528 27391 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:37:48.310194 27409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:48.310415 27391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:37:48.310532 27391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a"
format_stamp: "Formatted at 2026-08-06 22:37:47 on dist-test-slave-3kk6"
server_key: "5f8eacbb46623c0e471188c6db1aa820"
server_key_iv: "48fda6f4bfb5a3f4b3add8a593e26618"
server_key_version: "encryptionkey@0"
I20260806 22:37:48.311007 27391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:48.323683 27391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:48.325909 27391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:48.326089 27391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:48.326376 27391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:48.326830 27391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:48.326892 27391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:48.326939 27391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:48.326967 27391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:48.333129 27391 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:44843
I20260806 22:37:48.333214 27572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:44843 every 8 connection(s)
I20260806 22:37:48.333633 27391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:48.341018 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 27391
Aug 06 22:37:48 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055868, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.124@KRBTEST.COM
Aug 06 22:37:48 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055868, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
Aug 06 22:37:48 dist-test-slave-3kk6 krb5kdc[25752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055868, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.125@KRBTEST.COM
I20260806 22:37:48.347209 27577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.334518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.67:60847)
0806 22:37:48.334778 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.334826 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.335478 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.335800 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.335805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.336217 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:48.336409 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.336415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.337407 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.337408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.338491 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.338494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.339104 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.339105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.339295 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.339966 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:48.339975 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:48.342627 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:48.345315 (+ 2688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.345324 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.345327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.345720 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.345958 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.345959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.345960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.346101 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.346422 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:48.346429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:48.346618 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.346771 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.346962 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":68,"thread_start_us":90,"threads_started":1}
I20260806 22:37:48.347416 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.334882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.16.235.67:39371)
0806 22:37:48.335010 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.335021 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.335490 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.335867 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.335883 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.336194 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:48.336409 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.336414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.337265 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.337266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.338583 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.338586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.339583 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.339584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.339646 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.340294 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:48.340298 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:48.343163 (+ 2865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:48.345903 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.345904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.345905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.346127 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.346379 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.346380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.346380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.346440 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.347060 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:48.347061 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:48.347110 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.347178 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.347301 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":100,"thread_start_us":75,"threads_started":1}
I20260806 22:37:48.347695 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.334526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:57305 (local address 127.16.235.124:39341)
0806 22:37:48.334580 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.334582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.334939 (+ 357us) server_negotiation.cc:407] Connection header received
0806 22:37:48.335598 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.335600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.335619 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.335695 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.336534 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.337330 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.339119 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.339419 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.339498 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.343282 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.343284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.343284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.343292 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.345384 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.346845 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.346846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.346847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.346913 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.347136 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.347138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.347138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.347267 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.347321 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.347514 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.347605 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:37:48.347710 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.335029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:39371 (local address 127.16.235.125:43777)
0806 22:37:48.335079 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.335081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.335090 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:48.335776 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.335777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.335789 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.335842 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.336524 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.337086 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.338697 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.339003 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.339063 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.343274 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.343275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.343276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.343284 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.345812 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.346217 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.346218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.346219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.346303 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.346515 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.346516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.346516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.346965 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.347034 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.347544 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.347623 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:37:48.348062 27579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.334518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.16.235.67:57305)
0806 22:37:48.334817 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.334828 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.335478 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.336197 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.336198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.336365 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:48.336413 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.336415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.337418 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.337419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.338584 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.338586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.339500 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.339500 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.339550 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.340112 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:48.340117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:48.343163 (+ 3046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:48.345493 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.345494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.345495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.346737 (+ 1242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.346992 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:48.346992 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:48.346993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:48.347060 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:48.347356 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:48.347357 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:48.347394 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.347831 (+ 437us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.347953 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":230,"thread_start_us":64,"threads_started":1}
I20260806 22:37:48.348320 27575 heartbeater.cc:347] Connected to a master server at 127.16.235.125:43777
I20260806 22:37:48.348415 27575 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:48.348522 27573 heartbeater.cc:347] Connected to a master server at 127.16.235.124:39341
I20260806 22:37:48.348567 27573 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:48.348639 27575 heartbeater.cc:511] Master 127.16.235.125:43777 requested a full tablet report, sending...
I20260806 22:37:48.348639 27573 heartbeater.cc:511] Master 127.16.235.124:39341 requested a full tablet report, sending...
I20260806 22:37:48.349030 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.334587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:60847 (local address 127.16.235.126:44375)
0806 22:37:48.334677 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.334679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.334945 (+ 266us) server_negotiation.cc:407] Connection header received
0806 22:37:48.335687 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.335688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.335702 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.335762 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.336536 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.337305 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.338585 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.338957 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.339028 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.342778 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.342780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.342780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.342790 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.345110 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.345811 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.345812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.345813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.345878 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.346173 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.346174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.346174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.346317 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.346391 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.348813 (+ 2422us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.348913 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:37:48.349138 26934 ts_manager.cc:194] Registered new tserver with Master: 0e2e71ea1beb4efe8dff23d039ff0b5a (127.16.235.67:44843)
I20260806 22:37:48.349138 26647 ts_manager.cc:194] Registered new tserver with Master: 0e2e71ea1beb4efe8dff23d039ff0b5a (127.16.235.67:44843)
I20260806 22:37:48.349354 27574 heartbeater.cc:347] Connected to a master server at 127.16.235.126:44375
I20260806 22:37:48.349399 27574 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:48.349458 27574 heartbeater.cc:511] Master 127.16.235.126:44375 requested a full tablet report, sending...
I20260806 22:37:48.349773 25810 ts_manager.cc:194] Re-registered known tserver with Master: 0e2e71ea1beb4efe8dff23d039ff0b5a (127.16.235.67:44843)
I20260806 22:37:48.350239 25810 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:60847
I20260806 22:37:48.354151 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.342603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58956 (local address 127.16.235.124:39341)
0806 22:37:48.342738 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.342740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.342954 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:37:48.342982 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.342983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.342998 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.343060 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.344006 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.344862 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.345960 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.346265 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.346328 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.351000 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.351001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.351002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.351028 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.352783 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.353117 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.353119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.353119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.353190 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.353476 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.353477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.353478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.353637 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.353740 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.353843 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.353929 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260806 22:37:48.355914 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.342439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.16.235.125:43777)
0806 22:37:48.347785 (+ 5346us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.347788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.347797 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:48.347819 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.347820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.347830 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.347894 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.349409 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.350101 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.351089 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.351370 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.351456 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.352756 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.352757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.352758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.352765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.354571 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.355042 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.355043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.355043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.355152 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.355343 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.355344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.355344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.355499 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.355594 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.355723 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.355795 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5329,"thread_start_us":69,"threads_started":1}
I20260806 22:37:48.357419 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.344723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50040 (local address 127.16.235.126:44375)
0806 22:37:48.345656 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.345658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.345667 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:48.345688 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.345688 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.345700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.346723 (+ 1023us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.346765 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.350723 (+ 3958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.352007 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.352359 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.352434 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.354524 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.354525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.354526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.354534 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.356575 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.356865 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.356865 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.356866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.356917 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.357026 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.357026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.357026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.357115 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.357182 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.357251 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.357313 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":166,"threads_started":1}
I20260806 22:37:48.358852 25810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:48.360466 25810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:48.361315 26877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: NON_VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: true } } }
I20260806 22:37:48.361416 26877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:48.363116 27602 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: attempting to promote NON_VOTER 7eeb5d15afe14d7e9b257d568bd47440 to VOTER
I20260806 22:37:48.363284 27602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } }
I20260806 22:37:48.363755 26946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 LEARNER]: Refusing update from remote peer 047a87bf14f8429686f8d19bb1ec9fb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 22:37:48.363974 26664 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Refusing update from remote peer 047a87bf14f8429686f8d19bb1ec9fb6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 22:37:48.363976 27598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 22:37:48.364251 27598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 22:37:48.364713 27598 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7eeb5d15afe14d7e9b257d568bd47440 (127.16.235.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:48.365588 27598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:48.365669 27598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:48.365648 27602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:48.365790 27602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:48.365921 27598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:48.365983 27598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:48.365887 26946 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7eeb5d15afe14d7e9b257d568bd47440 (127.16.235.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:48.366828 26664 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7eeb5d15afe14d7e9b257d568bd47440 (127.16.235.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:48.367161 26993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:48.367278 26993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:48.367980 27614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:48.368075 27614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:48.373042 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.368841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54116 (local address 127.16.235.66:42963)
0806 22:37:48.369036 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.369042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.369052 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:37:48.369100 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.369102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.369116 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.369210 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:48.369330 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.370575 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.372101 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.372641 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.372750 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.372849 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.372911 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.372951 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260806 22:37:48.373065 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.368719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:42963 (local address 127.0.0.1:54116)
0806 22:37:48.368902 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.368910 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.368973 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.369205 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.369206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.369208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:48.369270 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.369273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.370681 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.370682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.372032 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.372034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.372755 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.372756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.372821 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.372826 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.372893 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.372940 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":208,"thread_start_us":117,"threads_started":1}
I20260806 22:37:48.373406 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.369122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34558 (local address 127.16.235.65:42679)
0806 22:37:48.369461 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.369468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.369480 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:37:48.369526 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.369528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.369545 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.369626 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:48.369739 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.370496 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.372021 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.372727 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.373121 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.373222 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.373255 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.373314 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":84,"threads_started":1}
I20260806 22:37:48.373521 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.368955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:44843 (local address 127.0.0.1:57762)
0806 22:37:48.369126 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.369138 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.369196 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.369801 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.369802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.369803 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:48.369838 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.369839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.371012 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.371012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.371798 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.371801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.373292 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.373293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.373350 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.373354 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.373401 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.373446 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260806 22:37:48.373926 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.369407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.16.235.67:44843)
0806 22:37:48.369634 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.369638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.369649 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:48.369687 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.369690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.369706 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.369802 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:48.369896 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.370916 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.371901 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.372663 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.372747 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.373790 (+ 1043us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.373822 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.373862 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":117,"threads_started":1}
I20260806 22:37:48.374030 27299 tablet_service.cc:1467] Processing CreateTablet for tablet e0bf38e32fe94e2eac8cef8d1e19ef39 (DEFAULT_TABLE table=test-table [id=7cfc792cdb50498db6989c5fb696c964]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:48.374452 27299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bf38e32fe94e2eac8cef8d1e19ef39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:48.377743 27624 tablet_bootstrap.cc:492] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: Bootstrap starting.
I20260806 22:37:48.378109 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.368651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:42679 (local address 127.0.0.1:34558)
0806 22:37:48.368701 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.368714 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.368754 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.369636 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.369637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.369639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:48.369681 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.369683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.370713 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.370713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.371928 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.371931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.372826 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.372827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.372904 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.372908 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.372938 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.372986 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":45}
I20260806 22:37:48.378749 27624 tablet_bootstrap.cc:654] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:48.378595 27481 tablet_service.cc:1467] Processing CreateTablet for tablet e0bf38e32fe94e2eac8cef8d1e19ef39 (DEFAULT_TABLE table=test-table [id=7cfc792cdb50498db6989c5fb696c964]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:48.379031 27481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bf38e32fe94e2eac8cef8d1e19ef39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:48.378906 27097 tablet_service.cc:1467] Processing CreateTablet for tablet e0bf38e32fe94e2eac8cef8d1e19ef39 (DEFAULT_TABLE table=test-table [id=7cfc792cdb50498db6989c5fb696c964]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:48.379194 27624 log.cc:824] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:48.379271 27097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0bf38e32fe94e2eac8cef8d1e19ef39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:48.380065 27624 tablet_bootstrap.cc:492] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: No bootstrap required, opened a new log
I20260806 22:37:48.380156 27624 ts_tablet_manager.cc:1397] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 22:37:48.381971 27624 raft_consensus.cc:348] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:48.382138 27624 raft_consensus.cc:374] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:48.382187 27624 raft_consensus.cc:729] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c6bb1251604c9dad9f23e814657bcf, State: Initialized, Role: FOLLOWER
I20260806 22:37:48.382313 27624 consensus_queue.cc:260] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:48.382740 27624 ts_tablet_manager.cc:1428] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:37:48.382851 27381 heartbeater.cc:503] Master 127.16.235.126:44375 was elected leader, sending a full tablet report...
I20260806 22:37:48.383837 27628 tablet_bootstrap.cc:492] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: Bootstrap starting.
I20260806 22:37:48.384132 27627 tablet_bootstrap.cc:492] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: Bootstrap starting.
I20260806 22:37:48.384735 27628 tablet_bootstrap.cc:654] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:48.385048 27628 log.cc:824] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:48.385088 27627 tablet_bootstrap.cc:654] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:48.385382 27627 log.cc:824] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:48.385929 27628 tablet_bootstrap.cc:492] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: No bootstrap required, opened a new log
I20260806 22:37:48.386029 27628 ts_tablet_manager.cc:1397] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:48.386207 27627 tablet_bootstrap.cc:492] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: No bootstrap required, opened a new log
I20260806 22:37:48.386279 27627 ts_tablet_manager.cc:1397] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:37:48.387794 27627 raft_consensus.cc:348] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:48.387830 27628 raft_consensus.cc:348] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:48.387956 27628 raft_consensus.cc:374] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:48.387960 27627 raft_consensus.cc:374] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:48.388013 27628 raft_consensus.cc:729] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2e71ea1beb4efe8dff23d039ff0b5a, State: Initialized, Role: FOLLOWER
I20260806 22:37:48.388016 27627 raft_consensus.cc:729] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bcee1edca234af1845705222b536dff, State: Initialized, Role: FOLLOWER
I20260806 22:37:48.388161 27627 consensus_queue.cc:260] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:48.388155 27628 consensus_queue.cc:260] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:48.388494 27627 ts_tablet_manager.cc:1428] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:37:48.388536 27628 ts_tablet_manager.cc:1428] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:37:48.388602 27574 heartbeater.cc:503] Master 127.16.235.126:44375 was elected leader, sending a full tablet report...
I20260806 22:37:48.388928 27189 heartbeater.cc:503] Master 127.16.235.126:44375 was elected leader, sending a full tablet report...
W20260806 22:37:48.419550 27387 tablet.cc:2369] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:48.482621 27191 tablet.cc:2369] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:48.584887 27576 tablet.cc:2369] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:48.718418 27633 raft_consensus.cc:482] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:48.718596 27633 raft_consensus.cc:504] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:48.719143 27633 leader_election.cc:288] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83c6bb1251604c9dad9f23e814657bcf (127.16.235.66:42963), 0e2e71ea1beb4efe8dff23d039ff0b5a (127.16.235.67:44843)
I20260806 22:37:48.722512 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.719721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:51801 (local address 127.16.235.67:44843)
0806 22:37:48.719814 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.719817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.719828 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:48.719917 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.719918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.719927 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.719970 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:48.720142 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.720792 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.721766 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.722133 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.722234 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.722242 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.722404 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.722440 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:37:48.722512 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.719482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:40925 (local address 127.16.235.66:42963)
0806 22:37:48.719556 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.719559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.719781 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:37:48.719818 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.719819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.719831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.719913 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:48.720062 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.720740 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.721762 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.722135 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.722224 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.722233 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.722393 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.722434 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:37:48.722532 27635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.719388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:42963 (local address 127.16.235.65:40925)
0806 22:37:48.719586 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.719598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.719709 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.719917 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.719919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.719922 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:48.719981 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.719983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.720836 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.720837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.721655 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.721657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.722224 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.722225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.722306 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.722315 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.722385 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.722434 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260806 22:37:48.722546 27636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.719620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:44843 (local address 127.16.235.65:51801)
0806 22:37:48.719715 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:48.719726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:48.719848 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:48.719983 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:48.719984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:48.719986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:48.720064 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.720066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.720876 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.720877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.721655 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:48.721657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.722235 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.722236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:48.722317 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.722320 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:48.722394 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:37:48.722437 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":44,"threads_started":1}
I20260806 22:37:48.722951 27517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bf38e32fe94e2eac8cef8d1e19ef39" candidate_uuid: "6bcee1edca234af1845705222b536dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" is_pre_election: true
I20260806 22:37:48.723006 27324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bf38e32fe94e2eac8cef8d1e19ef39" candidate_uuid: "6bcee1edca234af1845705222b536dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6bb1251604c9dad9f23e814657bcf" is_pre_election: true
I20260806 22:37:48.723141 27517 raft_consensus.cc:2454] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bcee1edca234af1845705222b536dff in term 0.
I20260806 22:37:48.723166 27324 raft_consensus.cc:2454] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bcee1edca234af1845705222b536dff in term 0.
I20260806 22:37:48.723412 27034 leader_election.cc:302] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bcee1edca234af1845705222b536dff, 83c6bb1251604c9dad9f23e814657bcf; no voters:
I20260806 22:37:48.723570 27633 raft_consensus.cc:2781] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:48.723637 27633 raft_consensus.cc:482] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:48.723661 27633 raft_consensus.cc:3037] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:48.724447 27633 raft_consensus.cc:504] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:48.724601 27633 leader_election.cc:288] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [CANDIDATE]: Term 1 election: Requested vote from peers 83c6bb1251604c9dad9f23e814657bcf (127.16.235.66:42963), 0e2e71ea1beb4efe8dff23d039ff0b5a (127.16.235.67:44843)
I20260806 22:37:48.724779 27517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bf38e32fe94e2eac8cef8d1e19ef39" candidate_uuid: "6bcee1edca234af1845705222b536dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a"
I20260806 22:37:48.724789 27324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0bf38e32fe94e2eac8cef8d1e19ef39" candidate_uuid: "6bcee1edca234af1845705222b536dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6bb1251604c9dad9f23e814657bcf"
I20260806 22:37:48.724865 27517 raft_consensus.cc:3037] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:48.724901 27324 raft_consensus.cc:3037] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:48.725739 27324 raft_consensus.cc:2454] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bcee1edca234af1845705222b536dff in term 1.
I20260806 22:37:48.725885 27517 raft_consensus.cc:2454] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bcee1edca234af1845705222b536dff in term 1.
I20260806 22:37:48.725982 27034 leader_election.cc:302] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bcee1edca234af1845705222b536dff, 83c6bb1251604c9dad9f23e814657bcf; no voters:
I20260806 22:37:48.726081 27633 raft_consensus.cc:2781] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:48.726210 27633 raft_consensus.cc:686] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 1 LEADER]: Becoming Leader. State: Replica: 6bcee1edca234af1845705222b536dff, State: Running, Role: LEADER
I20260806 22:37:48.726362 27633 consensus_queue.cc:237] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:48.727331 25809 catalog_manager.cc:5495] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bcee1edca234af1845705222b536dff (127.16.235.65). New cstate: current_term: 1 leader_uuid: "6bcee1edca234af1845705222b536dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:48.821380 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.818427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.16.235.65:42679)
0806 22:37:48.818520 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.818522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.818530 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:48.818569 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.818570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.818591 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.818695 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:48.818853 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.819433 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.820377 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.820697 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.820857 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.820999 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:48.821132 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:48.821232 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.821252 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.821275 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:37:48.823477 27517 raft_consensus.cc:1261] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 1 FOLLOWER]: Refusing update from remote peer 6bcee1edca234af1845705222b536dff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:48.823523 27324 raft_consensus.cc:1261] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [term 1 FOLLOWER]: Refusing update from remote peer 6bcee1edca234af1845705222b536dff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:48.823909 27633 consensus_queue.cc:1035] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:48.824108 27639 consensus_queue.cc:1035] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:48.827426 27643 mvcc.cc:205] Tried to move back new op lower bound from 7315684838698070016 to 7315684838303870976. Current Snapshot: MvccSnapshot[applied={T|T < 7315684838698070016}]
I20260806 22:37:48.827828 27642 mvcc.cc:205] Tried to move back new op lower bound from 7315684838698070016 to 7315684838303870976. Current Snapshot: MvccSnapshot[applied={T|T < 7315684838698070016}]
I20260806 22:37:48.831578 25810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:48.831713 25810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:48.833244 25810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 047a87bf14f8429686f8d19bb1ec9fb6: Sending DeleteTablet for 3 replicas of tablet e0bf38e32fe94e2eac8cef8d1e19ef39
I20260806 22:37:48.833648 27481 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bf38e32fe94e2eac8cef8d1e19ef39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:48 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:57762
I20260806 22:37:48.833662 27299 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bf38e32fe94e2eac8cef8d1e19ef39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:48 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:54116
I20260806 22:37:48.833808 27097 tablet_service.cc:1514] Processing DeleteTablet for tablet e0bf38e32fe94e2eac8cef8d1e19ef39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:48 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:34558
I20260806 22:37:48.834017 27655 tablet_replica.cc:330] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: stopping tablet replica
I20260806 22:37:48.834102 27656 tablet_replica.cc:330] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: stopping tablet replica
I20260806 22:37:48.834151 27655 raft_consensus.cc:2229] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:48.834195 27656 raft_consensus.cc:2229] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:48.834280 27655 raft_consensus.cc:2258] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:48.834326 27656 raft_consensus.cc:2258] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:48.834857 27656 ts_tablet_manager.cc:1905] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:48.834868 27655 ts_tablet_manager.cc:1905] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:48.836212 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 25771
I20260806 22:37:48.843111 27655 ts_tablet_manager.cc:1918] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:48.843221 27655 log.cc:1196] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/e0bf38e32fe94e2eac8cef8d1e19ef39
I20260806 22:37:48.843545 27655 ts_tablet_manager.cc:1939] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 83c6bb1251604c9dad9f23e814657bcf: Deleting consensus metadata
I20260806 22:37:48.843966 27656 ts_tablet_manager.cc:1918] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:48.844228 27656 log.cc:1196] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/e0bf38e32fe94e2eac8cef8d1e19ef39
I20260806 22:37:48.844630 27656 ts_tablet_manager.cc:1939] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 6bcee1edca234af1845705222b536dff: Deleting consensus metadata
I20260806 22:37:48.851447 27654 tablet_replica.cc:330] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: stopping tablet replica
I20260806 22:37:48.851672 27654 raft_consensus.cc:2229] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:48.851845 27654 raft_consensus.cc:2258] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:48.852449 27654 ts_tablet_manager.cc:1905] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 22:37:48.853267 27227 connection.cc:535] server connection from 127.0.0.1:54116 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:37:48.853737 27032 connection.cc:535] server connection from 127.0.0.1:34558 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:37:48.854506 27654 ts_tablet_manager.cc:1918] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:48.854593 27654 log.cc:1196] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/e0bf38e32fe94e2eac8cef8d1e19ef39
I20260806 22:37:48.855001 27654 ts_tablet_manager.cc:1939] T e0bf38e32fe94e2eac8cef8d1e19ef39 P 0e2e71ea1beb4efe8dff23d039ff0b5a: Deleting consensus metadata
W20260806 22:37:48.855404 27413 connection.cc:441] server connection from 127.0.0.1:57762 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57762 (request call id 1) could send its response
I20260806 22:37:48.860283 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.835822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.16.235.124:39341)
0806 22:37:48.835894 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.835897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.851299 (+ 15402us) server_negotiation.cc:407] Connection header received
0806 22:37:48.851355 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.851356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.851385 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.851469 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.852682 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.853498 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.854709 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.855058 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.855104 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.856976 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.856977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.856978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.856988 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.858971 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.859390 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.859391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.859391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.859450 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.859671 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.859672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.859672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.859800 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.859873 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.860067 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.860168 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:37:48.865046 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.856833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.16.235.125:43777)
0806 22:37:48.857207 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.857209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.857220 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:48.857246 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.857247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.857265 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.857331 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.857935 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.858762 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.859924 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.860244 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.860590 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.861751 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.861752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.861753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.861762 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.863688 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.864113 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.864114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.864114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.864183 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.864444 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.864445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.864445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.864590 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.864698 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.864840 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.864935 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":60,"threads_started":1}
I20260806 22:37:48.865649 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:48.857190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.16.235.124:39341)
0806 22:37:48.857290 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:48.857292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:48.857533 (+ 241us) server_negotiation.cc:407] Connection header received
0806 22:37:48.857558 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:48.857559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:48.857576 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:48.857637 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:48.858377 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.859181 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.860189 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:48.860502 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:48.860565 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:48.862176 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:48.862177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:48.862177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:48.862185 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:48.864152 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.864553 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.864554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.864554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.864619 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:48.864846 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:48.864847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:48.864847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:48.864985 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:48.865080 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:48.865478 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 22:37:48.865571 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":56,"threads_started":1}
I20260806 22:37:49.828773 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:49.828434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.65:36749)
0806 22:37:49.828595 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:49.828717 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260806 22:37:49.828765 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:49.828447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.67:47293)
0806 22:37:49.828621 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:49.828702 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
W20260806 22:37:49.829003 27574 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:44375 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
W20260806 22:37:49.829221 27189 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:44375 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
I20260806 22:37:49.829487 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:49.829151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.66:39859)
0806 22:37:49.829361 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:49.829442 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
W20260806 22:37:49.829748 27381 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:44375 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
I20260806 22:37:50.333736 27672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 047a87bf14f8429686f8d19bb1ec9fb6)
I20260806 22:37:50.333889 27672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } }
I20260806 22:37:50.334237 27672 leader_election.cc:288] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 047a87bf14f8429686f8d19bb1ec9fb6 (127.16.235.126:44375), 7eeb5d15afe14d7e9b257d568bd47440 (127.16.235.124:39341)
I20260806 22:37:50.334440 27673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 047a87bf14f8429686f8d19bb1ec9fb6)
I20260806 22:37:50.334626 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.334403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:50494)
0806 22:37:50.334518 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.334577 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":50,"threads_started":1}
I20260806 22:37:50.334640 27673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } }
W20260806 22:37:50.334826 26629 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
I20260806 22:37:50.335220 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.335131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:50498)
0806 22:37:50.335169 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.335196 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:37:50.335246 27680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.335037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:50496)
0806 22:37:50.335157 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.335206 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
W20260806 22:37:50.335399 26910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
I20260806 22:37:50.335629 27673 leader_election.cc:288] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 047a87bf14f8429686f8d19bb1ec9fb6 (127.16.235.126:44375), 85b2637944a540d3836f1591adba7a10 (127.16.235.125:43777)
W20260806 22:37:50.335633 26629 leader_election.cc:334] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 047a87bf14f8429686f8d19bb1ec9fb6 (127.16.235.126:44375): Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
I20260806 22:37:50.335894 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.335776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:50508)
0806 22:37:50.335813 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.335860 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 22:37:50.336076 26910 leader_election.cc:334] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 047a87bf14f8429686f8d19bb1ec9fb6 (127.16.235.126:44375): Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
I20260806 22:37:50.337795 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.334439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:39341 (local address 127.0.0.1:58982)
0806 22:37:50.334570 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.334583 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:50.334673 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:50.334859 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:50.334860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:50.334863 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:50.334927 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.334929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.335893 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.335894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.336833 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.336836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.337500 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.337501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.337567 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.337571 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:50.337649 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:37:50.337697 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":55,"threads_started":1}
I20260806 22:37:50.337836 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.334550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.16.235.124:39341)
0806 22:37:50.334690 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:50.334692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:50.334703 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:50.334740 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:50.334741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:50.334755 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:50.334839 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:50.335016 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.335808 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.336955 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.337399 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.337499 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.337510 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:50.337679 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:37:50.337749 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
I20260806 22:37:50.338203 26946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "85b2637944a540d3836f1591adba7a10" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7eeb5d15afe14d7e9b257d568bd47440" is_pre_election: true
I20260806 22:37:50.338344 26946 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85b2637944a540d3836f1591adba7a10 in term 1.
I20260806 22:37:50.338524 26631 leader_election.cc:302] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7eeb5d15afe14d7e9b257d568bd47440, 85b2637944a540d3836f1591adba7a10; no voters: 047a87bf14f8429686f8d19bb1ec9fb6
I20260806 22:37:50.338707 27672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:37:50.338752 27672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 047a87bf14f8429686f8d19bb1ec9fb6)
I20260806 22:37:50.338764 27680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.335242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:43777 (local address 127.0.0.1:59350)
0806 22:37:50.335313 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.335324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:50.335380 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:50.335943 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:50.335944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:50.335947 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:50.335990 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.335992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.336791 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.336792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.337737 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.337740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.338475 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.338476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.338547 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.338552 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:37:50.338613 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:37:50.338685 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":1,"thread_start_us":41,"threads_started":1}
I20260806 22:37:50.338795 27672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:37:50.338891 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.335465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.16.235.125:43777)
0806 22:37:50.335754 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:50.335757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:50.335765 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:50.335791 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:50.335792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:50.335830 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:50.335932 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:50.336077 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.336692 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.337837 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.338395 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.338485 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.338493 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:50.338678 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:37:50.338729 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":135,"threads_started":1}
I20260806 22:37:50.339727 27672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } }
I20260806 22:37:50.339886 27672 leader_election.cc:288] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [CANDIDATE]: Term 2 election: Requested vote from peers 047a87bf14f8429686f8d19bb1ec9fb6 (127.16.235.126:44375), 7eeb5d15afe14d7e9b257d568bd47440 (127.16.235.124:39341)
I20260806 22:37:50.340163 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.340053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:50524)
0806 22:37:50.340104 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.340137 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:37:50.340261 26946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "85b2637944a540d3836f1591adba7a10" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7eeb5d15afe14d7e9b257d568bd47440"
I20260806 22:37:50.340358 26946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:37:50.340358 26664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7eeb5d15afe14d7e9b257d568bd47440" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "85b2637944a540d3836f1591adba7a10" is_pre_election: true
I20260806 22:37:50.340462 26664 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7eeb5d15afe14d7e9b257d568bd47440 in current term 2: Already voted for candidate 85b2637944a540d3836f1591adba7a10 in this term.
I20260806 22:37:50.340541 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.340440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:50530)
0806 22:37:50.340490 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.340520 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:37:50.340610 26629 leader_election.cc:334] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 047a87bf14f8429686f8d19bb1ec9fb6 (127.16.235.126:44375): Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
I20260806 22:37:50.340754 26913 leader_election.cc:302] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7eeb5d15afe14d7e9b257d568bd47440; no voters: 047a87bf14f8429686f8d19bb1ec9fb6, 85b2637944a540d3836f1591adba7a10
I20260806 22:37:50.341229 26946 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85b2637944a540d3836f1591adba7a10 in term 2.
I20260806 22:37:50.341306 27673 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:37:50.341437 26631 leader_election.cc:302] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7eeb5d15afe14d7e9b257d568bd47440, 85b2637944a540d3836f1591adba7a10; no voters: 047a87bf14f8429686f8d19bb1ec9fb6
I20260806 22:37:50.341531 27672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:37:50.341732 27672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [term 2 LEADER]: Becoming Leader. State: Replica: 85b2637944a540d3836f1591adba7a10, State: Running, Role: LEADER
I20260806 22:37:50.341864 27672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } }
I20260806 22:37:50.342262 27685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85b2637944a540d3836f1591adba7a10. Latest consensus state: current_term: 2 leader_uuid: "85b2637944a540d3836f1591adba7a10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:50.342371 27685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:50.342568 27687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:50.343102 27687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:50.343194 27687 catalog_manager.cc:1259] Loaded cluster ID: dbff21308e2f421bbbd3035d967e73e1
I20260806 22:37:50.343233 27687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:50.343415 27687 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:50.343503 27687 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Loaded TSK: 0
I20260806 22:37:50.343652 27687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:37:50.356597 26647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:50.358242 26647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:50.359469 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.359334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:50556)
0806 22:37:50.359395 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.359435 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:37:50.359488 26946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [term 2 FOLLOWER]: Refusing update from remote peer 85b2637944a540d3836f1591adba7a10: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 22:37:50.359851 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.359720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.0.0.1:50564)
0806 22:37:50.359771 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.359799 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:37:50.359793 27685 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 22:37:50.359966 26629 consensus_peers.cc:487] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 -> Peer 047a87bf14f8429686f8d19bb1ec9fb6 (127.16.235.126:44375): Couldn't send request to peer 047a87bf14f8429686f8d19bb1ec9fb6. Status: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:37:50.360555 27673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85b2637944a540d3836f1591adba7a10. Latest consensus state: current_term: 2 leader_uuid: "85b2637944a540d3836f1591adba7a10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:50.360648 27673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:50.361167 27672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "85b2637944a540d3836f1591adba7a10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:50.361280 27672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:50.361352 27689 mvcc.cc:205] Tried to move back new op lower bound from 7315684844990222336 to 7315684844920999936. Current Snapshot: MvccSnapshot[applied={T|T < 7315684844990222336}]
I20260806 22:37:50.361388 27673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "85b2637944a540d3836f1591adba7a10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "047a87bf14f8429686f8d19bb1ec9fb6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 44375 } } peers { permanent_uuid: "85b2637944a540d3836f1591adba7a10" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 43777 } attrs { promote: false } } peers { permanent_uuid: "7eeb5d15afe14d7e9b257d568bd47440" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 39341 } attrs { promote: false } } }
I20260806 22:37:50.361500 27673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eeb5d15afe14d7e9b257d568bd47440 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:37:50.366889 27697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.363407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:42679 (local address 127.0.0.1:34564)
0806 22:37:50.363598 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.363611 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:50.363665 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:50.363931 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:50.363932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:50.363934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:50.363968 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.363969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.364864 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.364865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.365748 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.365751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.366564 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.366565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.366627 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.366633 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:37:50.366724 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 22:37:50.366767 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":68}
I20260806 22:37:50.367105 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.363325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:42963 (local address 127.0.0.1:54132)
0806 22:37:50.363573 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.363583 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:50.363649 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:50.363829 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:50.363830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:50.363831 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:50.363886 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.363888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.364888 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.364889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.366160 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.366163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.366898 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.366899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.366958 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.366962 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:50.366994 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 22:37:50.367032 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":114}
I20260806 22:37:50.367321 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.363570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.16.235.65:42679)
0806 22:37:50.363796 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:50.363809 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:50.363818 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:50.363849 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:50.363850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:50.363863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:50.363931 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:50.364051 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.364767 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.365877 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.366473 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.366547 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.366560 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:50.366886 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:37:50.367147 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":68,"threads_started":1}
I20260806 22:37:50.367357 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.363422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:44843 (local address 127.0.0.1:57764)
0806 22:37:50.363784 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.363794 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:50.363839 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:50.364185 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:50.364186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:50.364188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:50.364233 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.364234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.364933 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.364933 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.366160 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.366163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.367136 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.367137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.367200 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.367203 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:50.367258 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:37:50.367297 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":72,"thread_start_us":114,"threads_started":1}
I20260806 22:37:50.367377 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.363410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.16.235.66:42963)
0806 22:37:50.363552 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:50.363555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:50.363615 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:37:50.363715 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:50.363717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:50.363730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:50.363826 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:50.363966 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.364764 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.366268 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.366806 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.367223 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.367240 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:50.367261 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 22:37:50.367303 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260806 22:37:50.367569 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.363923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57764 (local address 127.16.235.67:44843)
0806 22:37:50.364041 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:50.364043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:50.364050 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:37:50.364076 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:50.364077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:50.364089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:50.364303 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:50.364340 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.364835 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.366247 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.366916 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.367097 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.367110 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:50.367452 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 22:37:50.367507 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260806 22:37:50.367565 27299 tablet_service.cc:1467] Processing CreateTablet for tablet 731a253f15b9424db628930bc175c2fb (DEFAULT_TABLE table=test-table [id=a08f2cb856a54ab0979fa754525fefcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:50.367570 27097 tablet_service.cc:1467] Processing CreateTablet for tablet 731a253f15b9424db628930bc175c2fb (DEFAULT_TABLE table=test-table [id=a08f2cb856a54ab0979fa754525fefcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:50.367709 27097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 731a253f15b9424db628930bc175c2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:50.367715 27299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 731a253f15b9424db628930bc175c2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:50.367939 27481 tablet_service.cc:1467] Processing CreateTablet for tablet 731a253f15b9424db628930bc175c2fb (DEFAULT_TABLE table=test-table [id=a08f2cb856a54ab0979fa754525fefcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:50.368052 27481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 731a253f15b9424db628930bc175c2fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:50.369329 27701 tablet_bootstrap.cc:492] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff: Bootstrap starting.
I20260806 22:37:50.369490 27702 tablet_bootstrap.cc:492] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: Bootstrap starting.
I20260806 22:37:50.369683 27701 tablet_bootstrap.cc:654] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:50.369933 27702 tablet_bootstrap.cc:654] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:50.370066 27700 tablet_bootstrap.cc:492] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: Bootstrap starting.
I20260806 22:37:50.370393 27700 tablet_bootstrap.cc:654] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:50.370610 27702 tablet_bootstrap.cc:492] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: No bootstrap required, opened a new log
I20260806 22:37:50.370692 27702 ts_tablet_manager.cc:1397] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:50.370863 27702 raft_consensus.cc:348] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:50.370922 27702 raft_consensus.cc:374] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:50.370936 27702 raft_consensus.cc:729] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2e71ea1beb4efe8dff23d039ff0b5a, State: Initialized, Role: FOLLOWER
I20260806 22:37:50.370970 27702 consensus_queue.cc:260] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:50.371085 27701 tablet_bootstrap.cc:492] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff: No bootstrap required, opened a new log
I20260806 22:37:50.371124 27701 ts_tablet_manager.cc:1397] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:50.371129 27702 ts_tablet_manager.cc:1428] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:50.371235 27701 raft_consensus.cc:348] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:50.371301 27701 raft_consensus.cc:374] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:50.371321 27701 raft_consensus.cc:729] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bcee1edca234af1845705222b536dff, State: Initialized, Role: FOLLOWER
I20260806 22:37:50.371366 27701 consensus_queue.cc:260] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:50.371519 27701 ts_tablet_manager.cc:1428] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:50.372197 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.371815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.65:43261)
0806 22:37:50.371968 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.372158 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260806 22:37:50.372663 27190 heartbeater.cc:503] Master 127.16.235.125:43777 was elected leader, sending a full tablet report...
I20260806 22:37:50.372786 27700 tablet_bootstrap.cc:492] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: No bootstrap required, opened a new log
I20260806 22:37:50.372848 27700 ts_tablet_manager.cc:1397] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:37:50.372977 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.372427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.65:40599)
0806 22:37:50.372922 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.372956 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475}
I20260806 22:37:50.373008 27700 raft_consensus.cc:348] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:50.373085 27700 raft_consensus.cc:374] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:50.373111 27700 raft_consensus.cc:729] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c6bb1251604c9dad9f23e814657bcf, State: Initialized, Role: FOLLOWER
I20260806 22:37:50.373164 27700 consensus_queue.cc:260] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:50.373328 27700 ts_tablet_manager.cc:1428] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:50.373706 27575 heartbeater.cc:503] Master 127.16.235.125:43777 was elected leader, sending a full tablet report...
I20260806 22:37:50.373900 27384 heartbeater.cc:503] Master 127.16.235.125:43777 was elected leader, sending a full tablet report...
I20260806 22:37:50.373944 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.371440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.67:45275)
0806 22:37:50.373860 (+ 2420us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.373902 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2394,"thread_start_us":114,"threads_started":1}
I20260806 22:37:50.374358 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.374167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.67:41343)
0806 22:37:50.374291 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.374335 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 22:37:50.374598 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.373496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.66:38957)
0806 22:37:50.374525 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.374564 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1004,"thread_start_us":91,"threads_started":1}
I20260806 22:37:50.374982 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.374895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.66:60343)
0806 22:37:50.374945 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.374966 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:37:50.395318 27706 raft_consensus.cc:482] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:50.395462 27706 raft_consensus.cc:504] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:50.395875 27706 leader_election.cc:288] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83c6bb1251604c9dad9f23e814657bcf (127.16.235.66:42963), 6bcee1edca234af1845705222b536dff (127.16.235.65:42679)
I20260806 22:37:50.399065 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.396179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:56341 (local address 127.16.235.65:42679)
0806 22:37:50.396248 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:50.396250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:50.396420 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:37:50.396455 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:50.396456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:50.396470 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:50.396540 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:50.396705 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.397162 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.398243 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.398626 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.398770 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.398784 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:50.398933 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:37:50.398989 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:37:50.399078 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.396067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:42963 (local address 127.16.235.67:59661)
0806 22:37:50.396137 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.396154 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:50.396239 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:50.396429 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:50.396431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:50.396437 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:50.396526 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.396529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.397307 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.397308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.398049 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.398051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.398796 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.398797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.398866 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.398869 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:50.398950 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:37:50.398989 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":1}
I20260806 22:37:50.399091 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.396193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:59661 (local address 127.16.235.66:42963)
0806 22:37:50.396246 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:50.396248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:50.396279 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:37:50.396311 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:50.396312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:50.396328 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:50.396390 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:50.396618 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.397194 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.398271 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.398619 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.398757 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.398768 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:50.398980 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 22:37:50.399027 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:37:50.399088 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.396067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:42679 (local address 127.16.235.67:56341)
0806 22:37:50.396347 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.396356 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:50.396408 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:50.396549 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:50.396549 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:50.396551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:50.396635 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.396637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.397247 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.397249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.398047 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:50.398050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.398773 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.398774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:50.398849 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.398853 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:50.398899 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:37:50.398954 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":92,"thread_start_us":132,"threads_started":1}
I20260806 22:37:50.399330 27324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731a253f15b9424db628930bc175c2fb" candidate_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6bb1251604c9dad9f23e814657bcf" is_pre_election: true
I20260806 22:37:50.399436 27324 raft_consensus.cc:2454] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2e71ea1beb4efe8dff23d039ff0b5a in term 0.
I20260806 22:37:50.399364 27131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731a253f15b9424db628930bc175c2fb" candidate_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bcee1edca234af1845705222b536dff" is_pre_election: true
I20260806 22:37:50.399515 27131 raft_consensus.cc:2454] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2e71ea1beb4efe8dff23d039ff0b5a in term 0.
I20260806 22:37:50.399632 27417 leader_election.cc:302] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2e71ea1beb4efe8dff23d039ff0b5a, 83c6bb1251604c9dad9f23e814657bcf; no voters:
I20260806 22:37:50.399801 27706 raft_consensus.cc:2781] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:50.399876 27706 raft_consensus.cc:482] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:50.399909 27706 raft_consensus.cc:3037] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:50.400687 27706 raft_consensus.cc:504] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:50.400852 27706 leader_election.cc:288] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [CANDIDATE]: Term 1 election: Requested vote from peers 83c6bb1251604c9dad9f23e814657bcf (127.16.235.66:42963), 6bcee1edca234af1845705222b536dff (127.16.235.65:42679)
I20260806 22:37:50.401069 27324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731a253f15b9424db628930bc175c2fb" candidate_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83c6bb1251604c9dad9f23e814657bcf"
I20260806 22:37:50.401157 27324 raft_consensus.cc:3037] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:50.401132 27131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731a253f15b9424db628930bc175c2fb" candidate_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bcee1edca234af1845705222b536dff"
I20260806 22:37:50.401217 27131 raft_consensus.cc:3037] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:50.401962 27324 raft_consensus.cc:2454] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2e71ea1beb4efe8dff23d039ff0b5a in term 1.
I20260806 22:37:50.402084 27131 raft_consensus.cc:2454] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2e71ea1beb4efe8dff23d039ff0b5a in term 1.
I20260806 22:37:50.402145 27417 leader_election.cc:302] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2e71ea1beb4efe8dff23d039ff0b5a, 83c6bb1251604c9dad9f23e814657bcf; no voters:
I20260806 22:37:50.402242 27706 raft_consensus.cc:2781] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:50.402418 27706 raft_consensus.cc:686] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 1 LEADER]: Becoming Leader. State: Replica: 0e2e71ea1beb4efe8dff23d039ff0b5a, State: Running, Role: LEADER
I20260806 22:37:50.402484 27706 consensus_queue.cc:237] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } }
I20260806 22:37:50.402976 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.402866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.67:53695)
0806 22:37:50.402909 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.402948 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:37:50.403394 26647 catalog_manager.cc:5495] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e2e71ea1beb4efe8dff23d039ff0b5a (127.16.235.67). New cstate: current_term: 1 leader_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e2e71ea1beb4efe8dff23d039ff0b5a" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 44843 } health_report { overall_health: HEALTHY } } }
I20260806 22:37:50.415731 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.413209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.16.235.67:44843)
0806 22:37:50.413367 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:50.413370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:50.413381 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:50.413426 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:50.413427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:50.413447 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:50.413524 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:50.413653 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.414147 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.415080 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:50.415270 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:50.415332 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:50.415460 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:50.415563 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:50.415618 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:50.415652 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:37:50.415671 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260806 22:37:50.416942 27324 raft_consensus.cc:1261] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [term 1 FOLLOWER]: Refusing update from remote peer 0e2e71ea1beb4efe8dff23d039ff0b5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:50.417068 27131 raft_consensus.cc:1261] T 731a253f15b9424db628930bc175c2fb P 6bcee1edca234af1845705222b536dff [term 1 FOLLOWER]: Refusing update from remote peer 0e2e71ea1beb4efe8dff23d039ff0b5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:50.417236 27706 consensus_queue.cc:1035] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "83c6bb1251604c9dad9f23e814657bcf" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 42963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:50.417421 27715 consensus_queue.cc:1035] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bcee1edca234af1845705222b536dff" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 42679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:50.418357 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.418232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.65:39993)
0806 22:37:50.418280 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.418315 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:37:50.418396 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.418277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.66:53895)
0806 22:37:50.418327 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.418358 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:37:50.419795 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.419670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.67:46689)
0806 22:37:50.419715 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.419760 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 22:37:50.420354 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.420238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.67:34681)
0806 22:37:50.420292 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.420320 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:37:50.420433 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.420322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.65:60407)
0806 22:37:50.420370 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.420397 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:37:50.420773 27721 mvcc.cc:205] Tried to move back new op lower bound from 7315684845225627648 to 7315684845169455104. Current Snapshot: MvccSnapshot[applied={T|T < 7315684845225627648}]
I20260806 22:37:50.421291 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.421178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:44375 (local address 127.16.235.66:33119)
0806 22:37:50.421231 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.421262 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:37:50.423856 26645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:50.424010 26645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:50.425462 26645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85b2637944a540d3836f1591adba7a10: Sending DeleteTablet for 3 replicas of tablet 731a253f15b9424db628930bc175c2fb
I20260806 22:37:50.425884 27481 tablet_service.cc:1514] Processing DeleteTablet for tablet 731a253f15b9424db628930bc175c2fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:50 UTC) from {username='oryx', principal='oryx/127.16.235.125@KRBTEST.COM'} at 127.0.0.1:57764
I20260806 22:37:50.425889 27299 tablet_service.cc:1514] Processing DeleteTablet for tablet 731a253f15b9424db628930bc175c2fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:50 UTC) from {username='oryx', principal='oryx/127.16.235.125@KRBTEST.COM'} at 127.0.0.1:54132
I20260806 22:37:50.426131 27732 tablet_replica.cc:330] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: stopping tablet replica
I20260806 22:37:50.426203 27732 raft_consensus.cc:2229] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:50.426308 27732 raft_consensus.cc:2258] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:50.426335 27097 tablet_service.cc:1514] Processing DeleteTablet for tablet 731a253f15b9424db628930bc175c2fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:50 UTC) from {username='oryx', principal='oryx/127.16.235.125@KRBTEST.COM'} at 127.0.0.1:34564
I20260806 22:37:50.426682 27732 ts_tablet_manager.cc:1905] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:50.426800 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 26873
I20260806 22:37:50.442267 27733 tablet_replica.cc:330] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: stopping tablet replica
I20260806 22:37:50.442415 27733 raft_consensus.cc:2229] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:50.442509 27733 raft_consensus.cc:2258] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:50.444729 27733 ts_tablet_manager.cc:1905] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:50.445722 27732 ts_tablet_manager.cc:1918] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:50.445814 27732 log.cc:1196] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/731a253f15b9424db628930bc175c2fb
I20260806 22:37:50.446095 27732 ts_tablet_manager.cc:1939] T 731a253f15b9424db628930bc175c2fb P 0e2e71ea1beb4efe8dff23d039ff0b5a: Deleting consensus metadata
I20260806 22:37:50.446560 26631 catalog_manager.cc:4841] TS 0e2e71ea1beb4efe8dff23d039ff0b5a (127.16.235.67:44843): tablet 731a253f15b9424db628930bc175c2fb (table test-table [id=a08f2cb856a54ab0979fa754525fefcb]) successfully deleted
I20260806 22:37:50.446707 27733 ts_tablet_manager.cc:1918] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:50.446779 27733 log.cc:1196] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/731a253f15b9424db628930bc175c2fb
I20260806 22:37:50.447011 27733 ts_tablet_manager.cc:1939] T 731a253f15b9424db628930bc175c2fb P 83c6bb1251604c9dad9f23e814657bcf: Deleting consensus metadata
I20260806 22:37:50.447440 26631 catalog_manager.cc:4841] TS 83c6bb1251604c9dad9f23e814657bcf (127.16.235.66:42963): tablet 731a253f15b9424db628930bc175c2fb (table test-table [id=a08f2cb856a54ab0979fa754525fefcb]) successfully deleted
I20260806 22:37:50.450806 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 27201
I20260806 22:37:50.450975 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:50.450808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:42679 (local address 127.0.0.1:34566)
0806 22:37:50.450897 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:50.450937 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:42679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 22:37:50.453584 26629 catalog_manager.cc:4584] TS 6bcee1edca234af1845705222b536dff (127.16.235.65:42679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 731a253f15b9424db628930bc175c2fb: Network error: Client connection negotiation failed: client connection to 127.16.235.65:42679: connect: Connection refused (error 111)
I20260806 22:37:50.462145 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 27391
I20260806 22:37:50.476490 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 26459
I20260806 22:37:50.489956 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 26726
2026-08-06T22:37:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5423 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:50 dist-test-slave-3kk6 krb5kdc[27738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:50 dist-test-slave-3kk6 krb5kdc[27738](info): set up 2 sockets
Aug 06 22:37:50 dist-test-slave-3kk6 krb5kdc[27738](info): commencing operation
krb5kdc: starting...
W20260806 22:37:51.736886 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:51 dist-test-slave-3kk6 krb5kdc[27738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.126.keytab.
Entry for principal oryx/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.126.keytab.
I20260806 22:37:51.871995 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:43797
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.126.keytab
--principal=oryx/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:45111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:43797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:51.961648 27757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:51.961903 27757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:51.961930 27757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:51.963187 27757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:51.963238 27757 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:37:51.963253 27757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:51.963266 27757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:51.963333 27757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:51.963358 27757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:51.965006 27757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:43797
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.126.keytab
--principal=oryx/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:43797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:51.965301 27757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:51.965586 27757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:51.970908 27771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:51.970940 27766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:51.971006 27765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:51.971032 27757 server_base.cc:1034] running on GCE node
I20260806 22:37:51.971498 27757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:51.971849 27757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:51.973033 27757 hybrid_clock.cc:648] HybridClock initialized: now 1786055871973022 us; error 58 us; skew 500 ppm
Aug 06 22:37:51 dist-test-slave-3kk6 krb5kdc[27738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055871, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:51.980449 27757 init.cc:377] Logged in from keytab as oryx/127.16.235.126@KRBTEST.COM (short username oryx)
I20260806 22:37:51.981035 27757 webserver.cc:460] Webserver started at http://127.16.235.126:42415/ using document root <none> and password file <none>
I20260806 22:37:51.981266 27757 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:51.981313 27757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:51.981431 27757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:51.982563 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "011fee1e76ab400a86ad8ac2b66dccf5"
format_stamp: "Formatted at 2026-08-06 22:37:51 on dist-test-slave-3kk6"
server_key: "1de6d9b0cc5f7712f8681efdf309a246"
server_key_iv: "7fde4a0e4c2c43eb4d76a8f17db56a75"
server_key_version: "encryptionkey@0"
I20260806 22:37:51.982990 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "011fee1e76ab400a86ad8ac2b66dccf5"
format_stamp: "Formatted at 2026-08-06 22:37:51 on dist-test-slave-3kk6"
server_key: "1de6d9b0cc5f7712f8681efdf309a246"
server_key_iv: "7fde4a0e4c2c43eb4d76a8f17db56a75"
server_key_version: "encryptionkey@0"
I20260806 22:37:51.984434 27757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:51.985239 27784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:51.985437 27757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:51.985541 27757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "011fee1e76ab400a86ad8ac2b66dccf5"
format_stamp: "Formatted at 2026-08-06 22:37:51 on dist-test-slave-3kk6"
server_key: "1de6d9b0cc5f7712f8681efdf309a246"
server_key_iv: "7fde4a0e4c2c43eb4d76a8f17db56a75"
server_key_version: "encryptionkey@0"
I20260806 22:37:51.985608 27757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:52.004607 27757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:52.007268 27757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:52.007408 27757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:52.012585 27757 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:43797
I20260806 22:37:52.012691 27884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:43797 every 8 connection(s)
I20260806 22:37:52.013078 27757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:52.013895 27886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:52.016388 27886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: Bootstrap starting.
I20260806 22:37:52.017004 27886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:52.017278 27886 log.cc:824] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:52.018004 27886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: No bootstrap required, opened a new log
I20260806 22:37:52.018929 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 27757
I20260806 22:37:52.019032 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:52.019248 17325 external_mini_cluster.cc:1428] Setting key 37ccf39ae6755d38d24234d7d923886c
I20260806 22:37:52.020965 27886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011fee1e76ab400a86ad8ac2b66dccf5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43797 } }
I20260806 22:37:52.021164 27886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:52.021206 27886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011fee1e76ab400a86ad8ac2b66dccf5, State: Initialized, Role: FOLLOWER
I20260806 22:37:52.021373 27886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011fee1e76ab400a86ad8ac2b66dccf5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43797 } }
I20260806 22:37:52.021473 27886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:52.021522 27886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:52.021574 27886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:52.022378 27886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011fee1e76ab400a86ad8ac2b66dccf5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43797 } }
I20260806 22:37:52.022503 27886 leader_election.cc:302] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 011fee1e76ab400a86ad8ac2b66dccf5; no voters:
I20260806 22:37:52.022719 27886 leader_election.cc:288] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:52.022792 27892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:52.022951 27892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [term 1 LEADER]: Becoming Leader. State: Replica: 011fee1e76ab400a86ad8ac2b66dccf5, State: Running, Role: LEADER
I20260806 22:37:52.023022 27886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:52.023061 27892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011fee1e76ab400a86ad8ac2b66dccf5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43797 } }
I20260806 22:37:52.023401 27894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "011fee1e76ab400a86ad8ac2b66dccf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011fee1e76ab400a86ad8ac2b66dccf5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43797 } } }
I20260806 22:37:52.023388 27892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 011fee1e76ab400a86ad8ac2b66dccf5. Latest consensus state: current_term: 1 leader_uuid: "011fee1e76ab400a86ad8ac2b66dccf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "011fee1e76ab400a86ad8ac2b66dccf5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 43797 } } }
I20260806 22:37:52.023576 27894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:52.023586 27892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5 [sys.catalog]: This master's current role is: LEADER
W20260806 22:37:52.024895 27913 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:37:52.024966 27913 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:37:52.025027 27904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:52.025220 27904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:52.027308 27904 catalog_manager.cc:1347] Generated new cluster ID: 13f7f019ea6c41b9897a32b3076da7a7
I20260806 22:37:52.027383 27904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:52.034953 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.019885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48724 (local address 127.16.235.126:43797)
0806 22:37:52.020122 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.020135 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.020183 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:37:52.020908 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.020920 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.020993 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.021166 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:52.022005 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.023430 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.024687 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.025354 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.025467 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.031257 (+ 5790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:52.031262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:52.031263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:52.031277 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:52.033206 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.033655 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.033658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.033658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.033758 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.033984 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.033985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.033986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.034256 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:52.034446 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.034534 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.034727 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260806 22:37:52.038823 27904 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:52.039566 27904 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:52.045440 27904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: Generated new TSK 0
I20260806 22:37:52.045676 27904 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab.
Entry for principal oryx/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab.
I20260806 22:37:52.077975 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:43797
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab
--principal=oryx/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:45111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:52.169895 27918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:52.170148 27918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:52.170176 27918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:52.171528 27918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:52.171582 27918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:52.171605 27918 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:37:52.171653 27918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:52.173308 27918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab
--principal=oryx/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.235.126:43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:52.173592 27918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:52.173854 27918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:52.179293 27930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:52.179315 27927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:52.179293 27926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:52.179549 27918 server_base.cc:1034] running on GCE node
I20260806 22:37:52.179790 27918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:52.180053 27918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:52.181214 27918 hybrid_clock.cc:648] HybridClock initialized: now 1786055872181192 us; error 42 us; skew 500 ppm
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055872, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:52.183984 27918 init.cc:377] Logged in from keytab as oryx/127.16.235.65@KRBTEST.COM (short username oryx)
I20260806 22:37:52.184484 27918 webserver.cc:460] Webserver started at http://127.16.235.65:34977/ using document root <none> and password file <none>
I20260806 22:37:52.184729 27918 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:52.184801 27918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:52.184965 27918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:52.186101 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "5abd7ffee617475c90484af02b2ebf79"
format_stamp: "Formatted at 2026-08-06 22:37:52 on dist-test-slave-3kk6"
server_key: "649d734769a46123104f622270df71c7"
server_key_iv: "cc6d2fd05bd03da365e03b8208d13b86"
server_key_version: "encryptionkey@0"
I20260806 22:37:52.186515 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "5abd7ffee617475c90484af02b2ebf79"
format_stamp: "Formatted at 2026-08-06 22:37:52 on dist-test-slave-3kk6"
server_key: "649d734769a46123104f622270df71c7"
server_key_iv: "cc6d2fd05bd03da365e03b8208d13b86"
server_key_version: "encryptionkey@0"
I20260806 22:37:52.188052 27918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:52.188895 27938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:52.189078 27918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:52.189193 27918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "5abd7ffee617475c90484af02b2ebf79"
format_stamp: "Formatted at 2026-08-06 22:37:52 on dist-test-slave-3kk6"
server_key: "649d734769a46123104f622270df71c7"
server_key_iv: "cc6d2fd05bd03da365e03b8208d13b86"
server_key_version: "encryptionkey@0"
I20260806 22:37:52.189275 27918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:52.201452 27918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:52.203641 27918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:52.203838 27918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:52.204438 27918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:52.204509 27918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:52.204547 27918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:52.204597 27918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055872, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:52.212242 27918 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:34241
I20260806 22:37:52.212333 28101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:34241 every 8 connection(s)
I20260806 22:37:52.212720 27918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:52.215201 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 27918
I20260806 22:37:52.215350 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:52.215525 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.204992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:43817 (local address 127.16.235.126:43797)
0806 22:37:52.205080 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.205083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.205334 (+ 251us) server_negotiation.cc:407] Connection header received
0806 22:37:52.205847 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.205849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.205864 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.205946 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:52.206917 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.207791 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.208928 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.209274 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.209344 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.212466 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:52.212468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:52.212468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:52.212478 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:52.214055 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.214464 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.214465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.214466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.214522 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.214812 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.214812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.214812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.214940 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:52.215037 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.215319 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.215423 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:37:52.215557 27957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.204998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:43797 (local address 127.16.235.65:43817)
0806 22:37:52.205186 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.205229 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.205740 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.206030 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.206033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.206454 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:52.206791 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.206799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.207919 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.207920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.208807 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.208810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.209365 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.209366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.209522 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.210217 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:52.210229 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:52.212336 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:52.214141 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.214148 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.214150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.214392 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.214583 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.214584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.214584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.214751 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.215028 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:52.215033 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:52.215160 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.215243 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.215381 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
I20260806 22:37:52.215583 17325 external_mini_cluster.cc:1428] Setting key 4eb7596d438e4b093a6548085af55bed
I20260806 22:37:52.216166 28102 heartbeater.cc:347] Connected to a master server at 127.16.235.126:43797
I20260806 22:37:52.216248 28102 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:52.216459 28102 heartbeater.cc:511] Master 127.16.235.126:43797 requested a full tablet report, sending...
I20260806 22:37:52.218943 27813 ts_manager.cc:194] Registered new tserver with Master: 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241)
I20260806 22:37:52.219805 27813 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:43817
I20260806 22:37:52.224835 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.217418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:53383 (local address 127.16.235.126:43797)
0806 22:37:52.217463 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.217465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.217528 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:37:52.217606 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.217606 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.217617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.217667 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:52.218009 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.218510 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.219624 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.219928 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.220004 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.221344 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:52.221345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:52.221345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:52.221352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:52.223288 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.223755 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.223756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.223756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.223814 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.224122 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.224123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.224123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.224354 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:52.224415 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.224620 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.224687 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:37:52.224862 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.217326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:43797 (local address 127.16.235.65:53383)
0806 22:37:52.217470 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.217481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.217547 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.217670 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.217671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.217870 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:52.217928 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.217930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.218648 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.218649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.219502 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.219505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.220066 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.220067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.220150 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.220733 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:52.220740 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:52.221242 (+ 502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:52.223395 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.223396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.223397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.223667 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.223928 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.223929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.223929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.224009 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.224439 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:52.224440 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:52.224502 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.224600 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.224723 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for oryx/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab.
Entry for principal oryx/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab.
I20260806 22:37:52.245522 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:43797
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab
--principal=oryx/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:45111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:52.332684 28113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:52.332978 28113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:52.333025 28113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:52.334314 28113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:52.334399 28113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:52.334435 28113 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:37:52.334478 28113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:52.336247 28113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab
--principal=oryx/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.235.126:43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:52.336582 28113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:52.336851 28113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:52.342139 28124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:52.342155 28118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:52.342376 28119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:52.342813 28113 server_base.cc:1034] running on GCE node
I20260806 22:37:52.343020 28113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:52.343252 28113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:52.344410 28113 hybrid_clock.cc:648] HybridClock initialized: now 1786055872344394 us; error 32 us; skew 500 ppm
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055872, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:52.347230 28113 init.cc:377] Logged in from keytab as oryx/127.16.235.66@KRBTEST.COM (short username oryx)
I20260806 22:37:52.347743 28113 webserver.cc:460] Webserver started at http://127.16.235.66:45595/ using document root <none> and password file <none>
I20260806 22:37:52.347973 28113 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:52.348016 28113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:52.348112 28113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:52.349187 28113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "2f476e38030d490f9c62b34e1d69460f"
format_stamp: "Formatted at 2026-08-06 22:37:52 on dist-test-slave-3kk6"
server_key: "e274375fb83b147b0088f7783f8d104e"
server_key_iv: "bff4f462dc465964ca6ffe026fb79ad1"
server_key_version: "encryptionkey@0"
I20260806 22:37:52.349501 28113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "2f476e38030d490f9c62b34e1d69460f"
format_stamp: "Formatted at 2026-08-06 22:37:52 on dist-test-slave-3kk6"
server_key: "e274375fb83b147b0088f7783f8d104e"
server_key_iv: "bff4f462dc465964ca6ffe026fb79ad1"
server_key_version: "encryptionkey@0"
I20260806 22:37:52.350971 28113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:37:52.351756 28134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:52.351940 28113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:52.352053 28113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "2f476e38030d490f9c62b34e1d69460f"
format_stamp: "Formatted at 2026-08-06 22:37:52 on dist-test-slave-3kk6"
server_key: "e274375fb83b147b0088f7783f8d104e"
server_key_iv: "bff4f462dc465964ca6ffe026fb79ad1"
server_key_version: "encryptionkey@0"
I20260806 22:37:52.352135 28113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:52.370913 28113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:52.373590 28113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:52.373791 28113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:52.374471 28113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:52.374528 28113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:52.374567 28113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:52.374588 28113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055872, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:52.382469 28113 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:44951
I20260806 22:37:52.382519 28297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:44951 every 8 connection(s)
I20260806 22:37:52.382939 28113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:52.385046 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.374981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:39887 (local address 127.16.235.126:43797)
0806 22:37:52.375060 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.375063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.375292 (+ 229us) server_negotiation.cc:407] Connection header received
0806 22:37:52.376023 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.376025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.376041 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.376127 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:52.376870 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.377408 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.378505 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.378851 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.378931 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.382034 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:52.382036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:52.382036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:52.382045 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:52.383843 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.384274 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.384275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.384275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.384335 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.384511 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.384511 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.384511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.384622 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:52.384669 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.384913 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.384980 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:37:52.385239 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.374951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:43797 (local address 127.16.235.66:39887)
0806 22:37:52.375143 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.375188 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.375891 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.376271 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.376278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.376540 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:52.376746 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.376751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.377516 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.377517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.378393 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.378396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.378941 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.378942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.379071 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.379705 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:52.379718 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:52.381869 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:52.383928 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.383951 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.383953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.384215 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.384379 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.384380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.384380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.384474 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.384676 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:52.384682 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:52.384809 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.384895 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.385029 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260806 22:37:52.385810 28298 heartbeater.cc:347] Connected to a master server at 127.16.235.126:43797
I20260806 22:37:52.385917 28298 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:52.386113 28298 heartbeater.cc:511] Master 127.16.235.126:43797 requested a full tablet report, sending...
I20260806 22:37:52.386600 27815 ts_manager.cc:194] Registered new tserver with Master: 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951)
I20260806 22:37:52.387198 27815 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:39887
I20260806 22:37:52.392378 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 28113
I20260806 22:37:52.392522 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:52.392784 17325 external_mini_cluster.cc:1428] Setting key c85e1d7592113e512aa2dd5215a73a64
I20260806 22:37:52.393060 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.387015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:45721 (local address 127.16.235.126:43797)
0806 22:37:52.387063 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.387065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.387125 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:37:52.387190 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.387190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.387199 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.387244 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:52.387636 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.388126 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.388921 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.389133 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.389223 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.390218 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:52.390220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:52.390220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:52.390228 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:52.391525 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.391938 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.391940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.391941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.392007 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.392260 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.392261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.392262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.392514 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:52.392593 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.392853 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.392958 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:37:52.393092 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.386919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:43797 (local address 127.16.235.66:45721)
0806 22:37:52.387062 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.387074 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.387134 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.387255 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.387256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.387489 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:52.387552 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.387554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.388228 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.388229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.388821 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.388823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.389225 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.389226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.389314 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.389764 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:52.389769 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:52.390090 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:52.391632 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.391634 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.391634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.391834 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.392080 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.392081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.392081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.392167 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.392625 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:52.392626 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:52.392715 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.392802 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.392940 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for oryx/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab.
Entry for principal oryx/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab.
I20260806 22:37:52.422730 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:43797
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab
--principal=oryx/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:45111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:52.510475 28312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:52.510741 28312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:52.510768 28312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:52.512053 28312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:52.512108 28312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:52.512130 28312 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:37:52.512180 28312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:52.513787 28312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab
--principal=oryx/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.235.126:43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:52.514057 28312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:52.514324 28312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:52.519838 28329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:52.519838 28327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:52.519850 28325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:52.520128 28312 server_base.cc:1034] running on GCE node
I20260806 22:37:52.520356 28312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:52.520618 28312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:52.521793 28312 hybrid_clock.cc:648] HybridClock initialized: now 1786055872521780 us; error 32 us; skew 500 ppm
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055872, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:52.525142 28312 init.cc:377] Logged in from keytab as oryx/127.16.235.67@KRBTEST.COM (short username oryx)
I20260806 22:37:52.525640 28312 webserver.cc:460] Webserver started at http://127.16.235.67:41589/ using document root <none> and password file <none>
I20260806 22:37:52.525902 28312 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:52.525979 28312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:52.526140 28312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:52.527410 28312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "f6c8bba496cf467ead48e8f745353554"
format_stamp: "Formatted at 2026-08-06 22:37:52 on dist-test-slave-3kk6"
server_key: "79cb70560417b0d5d0192a93a68378e8"
server_key_iv: "cb275b759e910d60e10450c157ad166e"
server_key_version: "encryptionkey@0"
I20260806 22:37:52.527835 28312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "f6c8bba496cf467ead48e8f745353554"
format_stamp: "Formatted at 2026-08-06 22:37:52 on dist-test-slave-3kk6"
server_key: "79cb70560417b0d5d0192a93a68378e8"
server_key_iv: "cb275b759e910d60e10450c157ad166e"
server_key_version: "encryptionkey@0"
I20260806 22:37:52.529525 28312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:52.530319 28337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:52.530508 28312 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:52.530624 28312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "f6c8bba496cf467ead48e8f745353554"
format_stamp: "Formatted at 2026-08-06 22:37:52 on dist-test-slave-3kk6"
server_key: "79cb70560417b0d5d0192a93a68378e8"
server_key_iv: "cb275b759e910d60e10450c157ad166e"
server_key_version: "encryptionkey@0"
I20260806 22:37:52.530761 28312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:52.545485 28312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:52.547950 28312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:52.548137 28312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:52.548732 28312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:52.548803 28312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:52.548856 28312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:52.548892 28312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:52.556492 28312 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:39391
I20260806 22:37:52.556618 28494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:39391 every 8 connection(s)
I20260806 22:37:52.557030 28312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055872, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:52.559389 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 28312
I20260806 22:37:52.559535 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:37:52.559749 17325 external_mini_cluster.cc:1428] Setting key 53e15a7c2e3d9afffa3300b98ca952c2
I20260806 22:37:52.561201 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.549380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:34715 (local address 127.16.235.126:43797)
0806 22:37:52.549428 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.549436 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.549719 (+ 283us) server_negotiation.cc:407] Connection header received
0806 22:37:52.550314 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.550315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.550329 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.550383 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:52.551453 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.552327 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.554168 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.554398 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.554459 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.557693 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:52.557695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:52.557695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:52.557705 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:52.559346 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.559911 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.559914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.559914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.559997 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.560441 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.560443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.560443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.560618 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:52.560697 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.561052 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.561135 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:37:52.561462 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.549397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:43797 (local address 127.16.235.67:34715)
0806 22:37:52.549555 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.549593 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.550169 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.550530 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.550535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.550995 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:52.551337 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.551345 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.552457 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.552458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.554028 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.554034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.554505 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.554506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.554709 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.555502 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:52.555516 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:52.557567 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:52.559468 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.559477 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.559480 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.559795 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.560159 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.560160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.560161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.560338 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.560687 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:52.560694 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:52.560922 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.561019 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.561172 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260806 22:37:52.561991 28496 heartbeater.cc:347] Connected to a master server at 127.16.235.126:43797
I20260806 22:37:52.562086 28496 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:52.562317 28496 heartbeater.cc:511] Master 127.16.235.126:43797 requested a full tablet report, sending...
I20260806 22:37:52.562747 27815 ts_manager.cc:194] Registered new tserver with Master: f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391)
I20260806 22:37:52.563203 27815 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:34715
I20260806 22:37:52.569103 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.563255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:46357 (local address 127.16.235.126:43797)
0806 22:37:52.563299 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.563301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.563533 (+ 232us) server_negotiation.cc:407] Connection header received
0806 22:37:52.563601 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.563602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.563612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.563679 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:52.563996 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.564527 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.565282 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.565471 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.565531 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.566568 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:52.566570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:52.566570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:52.566576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:52.568080 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.568420 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.568421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.568422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.568475 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.568689 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.568689 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.568690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.568798 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:52.568850 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.568973 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.569034 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:37:52.569129 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.563208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:43797 (local address 127.16.235.67:46357)
0806 22:37:52.563466 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.563480 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.563550 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.563681 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.563681 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.563852 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:52.563918 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.563920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.564640 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.564641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.565179 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.565181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.565543 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.565544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.565594 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.566150 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:52.566155 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:52.566480 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:52.568159 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.568160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.568161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.568350 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.568553 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:52.568554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:52.568554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:52.568625 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:52.568857 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:52.568858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:52.568900 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.568954 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.569048 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":179,"threads_started":1}
I20260806 22:37:52.571878 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:37:52.597033 27891 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:52.590565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48728 (local address 127.16.235.126:43797)
0806 22:37:52.590636 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.590639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.590676 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:37:52.590772 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.590773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.590784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.590849 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:52.591583 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.592222 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.593226 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.593419 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.593476 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.596918 (+ 3442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48728: BlockingRecv error: recv got EOF from 127.0.0.1:48728 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:37:52 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:52.605136 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.597955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48738 (local address 127.16.235.126:43797)
0806 22:37:52.598033 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.598035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.598139 (+ 104us) server_negotiation.cc:407] Connection header received
0806 22:37:52.598185 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.598186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.598196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.598250 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:52.598811 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.599301 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.600013 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.600188 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.600240 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.602502 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:52.602504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:52.602504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:52.602512 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:52.603938 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.604327 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.604328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.604329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.604392 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:52.604651 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:52.604652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:52.604653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:52.604781 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:52.604870 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.604996 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.605064 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:37:52.607038 27815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:52.608724 27815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:52.615727 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.611556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56678 (local address 127.16.235.66:44951)
0806 22:37:52.611791 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.611795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.611846 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:37:52.611937 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.611940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.611957 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.612037 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:52.612287 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.613424 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.614601 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.615241 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.615337 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.615426 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.615575 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.615616 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":120,"threads_started":1}
I20260806 22:37:52.615749 28523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.611287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:39391 (local address 127.0.0.1:38390)
0806 22:37:52.611757 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.611770 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.611839 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.612049 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.612052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.612055 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:52.612132 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.612134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.613121 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.613122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.614362 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.614365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.615455 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.615456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.615532 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.615537 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.615586 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.615632 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":116,"threads_started":1}
I20260806 22:37:52.616042 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.611279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:34241 (local address 127.0.0.1:47146)
0806 22:37:52.611484 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.611531 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.611606 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.612259 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.612260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.612262 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:52.612307 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.612308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.613555 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.613556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.614702 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.614705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.615677 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.615678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.615740 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.615745 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.615802 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.615996 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":28,"thread_start_us":76,"threads_started":1}
I20260806 22:37:52.616313 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.611509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47146 (local address 127.16.235.65:34241)
0806 22:37:52.611959 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.611963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.611971 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:52.612012 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.612014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.612038 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.612194 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:52.612389 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.613450 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.614773 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.615496 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.615600 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.615677 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.616171 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.616218 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":96,"threads_started":1}
I20260806 22:37:52.616322 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.611495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38390 (local address 127.16.235.67:39391)
0806 22:37:52.611711 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.611716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.611853 (+ 137us) server_negotiation.cc:407] Connection header received
0806 22:37:52.611907 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.611910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.611927 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.612020 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:52.612221 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.613016 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.614448 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.615259 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.615337 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.615417 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.616172 (+ 755us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.616218 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260806 22:37:52.616477 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.611290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:44951 (local address 127.0.0.1:56678)
0806 22:37:52.611775 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.611785 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.611856 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.612113 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.612115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.612117 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:52.612182 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.612184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.613538 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.613539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.614511 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.614513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.615340 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.615341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.615429 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.615446 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.615524 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.615588 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":34,"thread_start_us":100,"threads_started":1}
I20260806 22:37:52.617043 28403 tablet_service.cc:1467] Processing CreateTablet for tablet a35c1d32516f40c88fc5dffabeff4007 (DEFAULT_TABLE table=test-table [id=34685bbbdfcb4dc3991696c4bd38bec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:52.617045 28012 tablet_service.cc:1467] Processing CreateTablet for tablet a35c1d32516f40c88fc5dffabeff4007 (DEFAULT_TABLE table=test-table [id=34685bbbdfcb4dc3991696c4bd38bec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:52.617139 28206 tablet_service.cc:1467] Processing CreateTablet for tablet a35c1d32516f40c88fc5dffabeff4007 (DEFAULT_TABLE table=test-table [id=34685bbbdfcb4dc3991696c4bd38bec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:52.617432 28403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35c1d32516f40c88fc5dffabeff4007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:52.617440 28206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35c1d32516f40c88fc5dffabeff4007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:52.617436 28012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35c1d32516f40c88fc5dffabeff4007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:52.620590 28531 tablet_bootstrap.cc:492] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: Bootstrap starting.
I20260806 22:37:52.621277 28532 tablet_bootstrap.cc:492] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: Bootstrap starting.
I20260806 22:37:52.621562 28531 tablet_bootstrap.cc:654] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:52.621958 28532 tablet_bootstrap.cc:654] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:52.621922 28531 log.cc:824] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:52.622293 28532 log.cc:824] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:52.623008 28531 tablet_bootstrap.cc:492] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: No bootstrap required, opened a new log
I20260806 22:37:52.623126 28532 tablet_bootstrap.cc:492] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: No bootstrap required, opened a new log
I20260806 22:37:52.623154 28531 ts_tablet_manager.cc:1397] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:37:52.623199 28532 ts_tablet_manager.cc:1397] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:52.624609 28535 tablet_bootstrap.cc:492] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: Bootstrap starting.
I20260806 22:37:52.624994 28532 raft_consensus.cc:348] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } }
I20260806 22:37:52.625195 28532 raft_consensus.cc:374] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:52.625247 28532 raft_consensus.cc:729] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5abd7ffee617475c90484af02b2ebf79, State: Initialized, Role: FOLLOWER
I20260806 22:37:52.625280 28531 raft_consensus.cc:348] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } }
I20260806 22:37:52.625468 28531 raft_consensus.cc:374] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:52.625417 28532 consensus_queue.cc:260] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } }
I20260806 22:37:52.625514 28535 tablet_bootstrap.cc:654] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:52.625516 28531 raft_consensus.cc:729] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c8bba496cf467ead48e8f745353554, State: Initialized, Role: FOLLOWER
I20260806 22:37:52.625747 28532 ts_tablet_manager.cc:1428] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:37:52.625825 28102 heartbeater.cc:503] Master 127.16.235.126:43797 was elected leader, sending a full tablet report...
I20260806 22:37:52.625864 28535 log.cc:824] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:52.626210 28531 consensus_queue.cc:260] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } }
I20260806 22:37:52.626529 28531 ts_tablet_manager.cc:1428] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:37:52.626619 28496 heartbeater.cc:503] Master 127.16.235.126:43797 was elected leader, sending a full tablet report...
I20260806 22:37:52.626858 28535 tablet_bootstrap.cc:492] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: No bootstrap required, opened a new log
I20260806 22:37:52.626947 28535 ts_tablet_manager.cc:1397] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:52.628559 28535 raft_consensus.cc:348] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } }
I20260806 22:37:52.628697 28535 raft_consensus.cc:374] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:52.628721 28535 raft_consensus.cc:729] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f476e38030d490f9c62b34e1d69460f, State: Initialized, Role: FOLLOWER
I20260806 22:37:52.628842 28535 consensus_queue.cc:260] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } }
I20260806 22:37:52.629169 28535 ts_tablet_manager.cc:1428] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:37:52.629343 28298 heartbeater.cc:503] Master 127.16.235.126:43797 was elected leader, sending a full tablet report...
W20260806 22:37:52.633559 28299 tablet.cc:2369] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:52.713908 28103 tablet.cc:2369] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:37:52.807708 28497 tablet.cc:2369] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:37:52.853741 28540 raft_consensus.cc:482] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:52.853893 28540 raft_consensus.cc:504] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } }
I20260806 22:37:52.854327 28540 leader_election.cc:288] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241), f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391)
I20260806 22:37:52.858108 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.854773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:57489 (local address 127.16.235.65:34241)
0806 22:37:52.854824 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.854826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.854835 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:52.854870 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.854871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.854882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.854940 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:52.855235 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.855717 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.857135 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.857713 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.857799 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.857811 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.858002 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.858044 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:37:52.858108 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.854803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:39871 (local address 127.16.235.67:39391)
0806 22:37:52.854868 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.854870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.854881 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:52.854945 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.854946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.854956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.855011 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:52.855218 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.855991 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.857236 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.857783 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.857858 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.857865 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.857997 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.858036 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:37:52.858139 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.854598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:34241 (local address 127.16.235.66:57489)
0806 22:37:52.854687 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.854700 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.854755 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.855060 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.855061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.855063 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:52.855140 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.855141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.855808 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.855810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.857035 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.857038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.857803 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.857803 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.857909 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.857914 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.857975 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.858022 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":22}
I20260806 22:37:52.858146 28542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.854634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:39391 (local address 127.16.235.66:39871)
0806 22:37:52.854821 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:52.854832 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:52.854895 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:52.855038 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:52.855039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:52.855041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:52.855137 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.855140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.856088 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.856089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.857151 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:52.857153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.857861 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.857862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:52.857920 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.857923 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:52.857975 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:37:52.858022 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":2,"thread_start_us":116,"threads_started":1}
I20260806 22:37:52.858687 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35c1d32516f40c88fc5dffabeff4007" candidate_uuid: "2f476e38030d490f9c62b34e1d69460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5abd7ffee617475c90484af02b2ebf79" is_pre_election: true
I20260806 22:37:52.858781 28431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35c1d32516f40c88fc5dffabeff4007" candidate_uuid: "2f476e38030d490f9c62b34e1d69460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c8bba496cf467ead48e8f745353554" is_pre_election: true
I20260806 22:37:52.858886 28037 raft_consensus.cc:2454] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f476e38030d490f9c62b34e1d69460f in term 0.
I20260806 22:37:52.858950 28431 raft_consensus.cc:2454] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f476e38030d490f9c62b34e1d69460f in term 0.
I20260806 22:37:52.859151 28135 leader_election.cc:302] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f476e38030d490f9c62b34e1d69460f, 5abd7ffee617475c90484af02b2ebf79; no voters:
I20260806 22:37:52.859311 28540 raft_consensus.cc:2781] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:52.859405 28540 raft_consensus.cc:482] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:52.859441 28540 raft_consensus.cc:3037] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:52.860216 28540 raft_consensus.cc:504] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } }
I20260806 22:37:52.860374 28540 leader_election.cc:288] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [CANDIDATE]: Term 1 election: Requested vote from peers 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241), f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391)
I20260806 22:37:52.860526 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35c1d32516f40c88fc5dffabeff4007" candidate_uuid: "2f476e38030d490f9c62b34e1d69460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5abd7ffee617475c90484af02b2ebf79"
I20260806 22:37:52.860628 28037 raft_consensus.cc:3037] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:52.860633 28431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35c1d32516f40c88fc5dffabeff4007" candidate_uuid: "2f476e38030d490f9c62b34e1d69460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c8bba496cf467ead48e8f745353554"
I20260806 22:37:52.860728 28431 raft_consensus.cc:3037] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:52.861404 28037 raft_consensus.cc:2454] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f476e38030d490f9c62b34e1d69460f in term 1.
I20260806 22:37:52.861481 28431 raft_consensus.cc:2454] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f476e38030d490f9c62b34e1d69460f in term 1.
I20260806 22:37:52.861624 28135 leader_election.cc:302] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f476e38030d490f9c62b34e1d69460f, 5abd7ffee617475c90484af02b2ebf79; no voters:
I20260806 22:37:52.861768 28540 raft_consensus.cc:2781] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:52.861943 28540 raft_consensus.cc:686] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 1 LEADER]: Becoming Leader. State: Replica: 2f476e38030d490f9c62b34e1d69460f, State: Running, Role: LEADER
I20260806 22:37:52.862066 28540 consensus_queue.cc:237] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } }
I20260806 22:37:52.863015 27813 catalog_manager.cc:5495] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f476e38030d490f9c62b34e1d69460f (127.16.235.66). New cstate: current_term: 1 leader_uuid: "2f476e38030d490f9c62b34e1d69460f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } health_report { overall_health: HEALTHY } } }
I20260806 22:37:52.907140 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:52.904819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56684 (local address 127.16.235.66:44951)
0806 22:37:52.904880 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:52.904883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:52.904892 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:52.904940 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:52.904941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:52.904959 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:52.905021 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:52.905171 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.905640 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.906412 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:52.906587 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:52.906636 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:52.906853 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:52.906953 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:52.907017 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:52.907063 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:37:52.907081 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:37:52.909036 28037 raft_consensus.cc:1261] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Refusing update from remote peer 2f476e38030d490f9c62b34e1d69460f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:52.909075 28431 raft_consensus.cc:1261] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Refusing update from remote peer 2f476e38030d490f9c62b34e1d69460f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:52.909412 28540 consensus_queue.cc:1035] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:52.909569 28543 consensus_queue.cc:1035] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:52.912160 28548 mvcc.cc:205] Tried to move back new op lower bound from 7315684855432282112 to 7315684855243923456. Current Snapshot: MvccSnapshot[applied={T|T < 7315684855432282112}]
I20260806 22:37:52.914644 27815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:52.914800 27815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:52.915582 27815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: Sending DeleteTablet for 3 replicas of tablet a35c1d32516f40c88fc5dffabeff4007
I20260806 22:37:52.915920 28206 tablet_service.cc:1514] Processing DeleteTablet for tablet a35c1d32516f40c88fc5dffabeff4007 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:52 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:56678
I20260806 22:37:52.915917 28403 tablet_service.cc:1514] Processing DeleteTablet for tablet a35c1d32516f40c88fc5dffabeff4007 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:52 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:38390
I20260806 22:37:52.916076 28012 tablet_service.cc:1514] Processing DeleteTablet for tablet a35c1d32516f40c88fc5dffabeff4007 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:52 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47146
I20260806 22:37:52.916188 28565 tablet_replica.cc:330] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: stopping tablet replica
I20260806 22:37:52.916290 28565 raft_consensus.cc:2229] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:52.916420 28565 raft_consensus.cc:2258] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:52.916595 28566 tablet_replica.cc:330] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: stopping tablet replica
I20260806 22:37:52.916658 27815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:52.916800 27815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:52.916970 28565 ts_tablet_manager.cc:1905] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:52.916985 28567 tablet_replica.cc:330] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: stopping tablet replica
I20260806 22:37:52.917093 28567 raft_consensus.cc:2229] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:52.917022 28566 raft_consensus.cc:2229] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:52.917725 28566 raft_consensus.cc:2258] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:52.918148 28012 tablet_service.cc:1467] Processing CreateTablet for tablet ebb66f3ae60c42f8be2b6f406a161d9a (DEFAULT_TABLE table=test-table [id=5d3b0a995ccc4f609b7490558c42afce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:52.918282 28012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebb66f3ae60c42f8be2b6f406a161d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:52.918476 28566 ts_tablet_manager.cc:1905] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:52.919122 28565 ts_tablet_manager.cc:1918] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:52.919226 28565 log.cc:1196] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/a35c1d32516f40c88fc5dffabeff4007
I20260806 22:37:52.919351 28532 tablet_bootstrap.cc:492] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: Bootstrap starting.
I20260806 22:37:52.919566 28565 ts_tablet_manager.cc:1939] T a35c1d32516f40c88fc5dffabeff4007 P 2f476e38030d490f9c62b34e1d69460f: Deleting consensus metadata
I20260806 22:37:52.919791 28532 tablet_bootstrap.cc:654] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:52.920028 27791 catalog_manager.cc:4841] TS 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951): tablet a35c1d32516f40c88fc5dffabeff4007 (table test-table [id=34685bbbdfcb4dc3991696c4bd38bec6]) successfully deleted
I20260806 22:37:52.920400 28566 ts_tablet_manager.cc:1918] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:52.920476 28403 tablet_service.cc:1467] Processing CreateTablet for tablet ebb66f3ae60c42f8be2b6f406a161d9a (DEFAULT_TABLE table=test-table [id=5d3b0a995ccc4f609b7490558c42afce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:52.920485 28566 log.cc:1196] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/a35c1d32516f40c88fc5dffabeff4007
I20260806 22:37:52.920531 28206 tablet_service.cc:1467] Processing CreateTablet for tablet ebb66f3ae60c42f8be2b6f406a161d9a (DEFAULT_TABLE table=test-table [id=5d3b0a995ccc4f609b7490558c42afce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:52.920622 28206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebb66f3ae60c42f8be2b6f406a161d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:52.920624 28403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebb66f3ae60c42f8be2b6f406a161d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:52.920837 28566 ts_tablet_manager.cc:1939] T a35c1d32516f40c88fc5dffabeff4007 P f6c8bba496cf467ead48e8f745353554: Deleting consensus metadata
I20260806 22:37:52.921842 28531 tablet_bootstrap.cc:492] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: Bootstrap starting.
I20260806 22:37:52.922025 27790 catalog_manager.cc:4841] TS f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391): tablet a35c1d32516f40c88fc5dffabeff4007 (table test-table [id=34685bbbdfcb4dc3991696c4bd38bec6]) successfully deleted
I20260806 22:37:52.922288 28531 tablet_bootstrap.cc:654] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:52.922359 28532 tablet_bootstrap.cc:492] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: No bootstrap required, opened a new log
I20260806 22:37:52.922415 28532 ts_tablet_manager.cc:1397] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:37:52.922613 28532 raft_consensus.cc:348] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:52.922713 28532 raft_consensus.cc:374] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:52.922741 28532 raft_consensus.cc:729] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5abd7ffee617475c90484af02b2ebf79, State: Initialized, Role: FOLLOWER
I20260806 22:37:52.922730 28535 tablet_bootstrap.cc:492] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: Bootstrap starting.
I20260806 22:37:52.922791 28532 consensus_queue.cc:260] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:52.922904 28532 ts_tablet_manager.cc:1428] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:52.923177 28535 tablet_bootstrap.cc:654] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:52.923190 28567 raft_consensus.cc:2258] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:52.923913 28567 ts_tablet_manager.cc:1905] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:52.923936 28535 tablet_bootstrap.cc:492] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: No bootstrap required, opened a new log
I20260806 22:37:52.923992 28535 ts_tablet_manager.cc:1397] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:52.924149 28535 raft_consensus.cc:348] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:52.924233 28535 raft_consensus.cc:374] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:52.924259 28535 raft_consensus.cc:729] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f476e38030d490f9c62b34e1d69460f, State: Initialized, Role: FOLLOWER
I20260806 22:37:52.924312 28535 consensus_queue.cc:260] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:52.924417 28535 ts_tablet_manager.cc:1428] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:52.924975 28531 tablet_bootstrap.cc:492] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: No bootstrap required, opened a new log
I20260806 22:37:52.925029 28531 ts_tablet_manager.cc:1397] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:37:52.925187 28531 raft_consensus.cc:348] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:52.925257 28531 raft_consensus.cc:374] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:52.925280 28531 raft_consensus.cc:729] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c8bba496cf467ead48e8f745353554, State: Initialized, Role: FOLLOWER
I20260806 22:37:52.925328 28531 consensus_queue.cc:260] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:52.925414 28531 ts_tablet_manager.cc:1428] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:52.926083 28567 ts_tablet_manager.cc:1918] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:52.926174 28567 log.cc:1196] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/a35c1d32516f40c88fc5dffabeff4007
I20260806 22:37:52.926434 28567 ts_tablet_manager.cc:1939] T a35c1d32516f40c88fc5dffabeff4007 P 5abd7ffee617475c90484af02b2ebf79: Deleting consensus metadata
I20260806 22:37:52.926885 27787 catalog_manager.cc:4841] TS 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241): tablet a35c1d32516f40c88fc5dffabeff4007 (table test-table [id=34685bbbdfcb4dc3991696c4bd38bec6]) successfully deleted
I20260806 22:37:53.103305 28539 raft_consensus.cc:482] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:53.103436 28539 raft_consensus.cc:504] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.103873 28539 leader_election.cc:288] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241), 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951)
I20260806 22:37:53.107244 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.104172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:45715 (local address 127.16.235.66:44951)
0806 22:37:53.104227 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.104230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.104330 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:37:53.104432 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.104433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.104448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.104504 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:53.104673 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.105173 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.106170 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.106835 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.106907 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.106915 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.107119 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.107165 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:37:53.107270 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.104295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:42261 (local address 127.16.235.65:34241)
0806 22:37:53.104343 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.104345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.104355 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:37:53.104465 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.104466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.104476 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.104535 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:53.104724 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.105181 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.106245 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.106864 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.106982 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.106991 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.107159 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.107201 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:37:53.107285 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.104132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:34241 (local address 127.16.235.67:42261)
0806 22:37:53.104290 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.104302 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.104367 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.104555 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.104556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.104558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:53.104627 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.104628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.105285 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.105286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.106143 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.106146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.107004 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.107005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.107059 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.107081 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.107141 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.107178 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260806 22:37:53.107283 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.104088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:44951 (local address 127.16.235.67:45715)
0806 22:37:53.104247 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.104261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.104365 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.104510 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.104511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.104513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:53.104599 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.104601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.105262 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.105263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.106077 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.106079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.106947 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.106948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.107027 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.107031 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.107086 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.107141 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":34,"thread_start_us":89,"threads_started":1}
I20260806 22:37:53.107607 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebb66f3ae60c42f8be2b6f406a161d9a" candidate_uuid: "f6c8bba496cf467ead48e8f745353554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5abd7ffee617475c90484af02b2ebf79" is_pre_election: true
I20260806 22:37:53.107627 28234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebb66f3ae60c42f8be2b6f406a161d9a" candidate_uuid: "f6c8bba496cf467ead48e8f745353554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f476e38030d490f9c62b34e1d69460f" is_pre_election: true
I20260806 22:37:53.107739 28037 raft_consensus.cc:2454] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6c8bba496cf467ead48e8f745353554 in term 0.
I20260806 22:37:53.107788 28234 raft_consensus.cc:2454] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6c8bba496cf467ead48e8f745353554 in term 0.
I20260806 22:37:53.107963 28339 leader_election.cc:302] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5abd7ffee617475c90484af02b2ebf79, f6c8bba496cf467ead48e8f745353554; no voters:
I20260806 22:37:53.108119 28539 raft_consensus.cc:2781] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:53.108179 28539 raft_consensus.cc:482] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:53.108223 28539 raft_consensus.cc:3037] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.108974 28539 raft_consensus.cc:504] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.109126 28539 leader_election.cc:288] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [CANDIDATE]: Term 1 election: Requested vote from peers 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241), 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951)
I20260806 22:37:53.109282 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebb66f3ae60c42f8be2b6f406a161d9a" candidate_uuid: "f6c8bba496cf467ead48e8f745353554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5abd7ffee617475c90484af02b2ebf79"
I20260806 22:37:53.109316 28234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebb66f3ae60c42f8be2b6f406a161d9a" candidate_uuid: "f6c8bba496cf467ead48e8f745353554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f476e38030d490f9c62b34e1d69460f"
I20260806 22:37:53.109371 28234 raft_consensus.cc:3037] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.109388 28037 raft_consensus.cc:3037] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.110036 28234 raft_consensus.cc:2454] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6c8bba496cf467ead48e8f745353554 in term 1.
I20260806 22:37:53.110105 28037 raft_consensus.cc:2454] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6c8bba496cf467ead48e8f745353554 in term 1.
I20260806 22:37:53.110258 28339 leader_election.cc:302] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5abd7ffee617475c90484af02b2ebf79, f6c8bba496cf467ead48e8f745353554; no voters:
I20260806 22:37:53.110371 28539 raft_consensus.cc:2781] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:53.110538 28539 raft_consensus.cc:686] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 1 LEADER]: Becoming Leader. State: Replica: f6c8bba496cf467ead48e8f745353554, State: Running, Role: LEADER
I20260806 22:37:53.110613 28539 consensus_queue.cc:237] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.111352 27815 catalog_manager.cc:5495] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6c8bba496cf467ead48e8f745353554 (127.16.235.67). New cstate: current_term: 1 leader_uuid: "f6c8bba496cf467ead48e8f745353554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } health_report { overall_health: HEALTHY } } }
Aug 06 22:37:53 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055871, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.126@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:53.187049 28584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.179565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:43797 (local address 127.0.0.1:48748)
0806 22:37:53.179760 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.179775 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.179864 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.180012 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.180012 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.180347 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:53.180400 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.180402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.181055 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.181056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.181589 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.181590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.181915 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.181916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.181956 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.182485 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:53.182490 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:53.183962 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:53.185785 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:53.185792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:53.185794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:53.185991 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:53.186220 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:53.186222 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:53.186222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:53.186306 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:53.186552 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:53.186559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:53.186609 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.186767 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.186918 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260806 22:37:53.187062 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.179653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.16.235.126:43797)
0806 22:37:53.179849 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.179851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.179862 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:53.179918 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.179919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.179928 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.179993 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:53.180470 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.180985 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.181660 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.181843 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.181899 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.184050 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:53.184052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:53.184052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:53.184058 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:53.185659 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:53.186056 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:53.186057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:53.186058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:53.186119 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:53.186377 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:53.186378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:53.186378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:53.186492 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:53.186546 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.186805 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.186982 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":172,"thread_start_us":138,"threads_started":1}
I20260806 22:37:53.189108 27815 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:48748:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:37:53.189361 27815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:53.190948 28403 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef08a9e7e974b229df7b30cdb0de23f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=54b193c0f234417593bf3b8fd9920419]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:37:53.190958 28012 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef08a9e7e974b229df7b30cdb0de23f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=54b193c0f234417593bf3b8fd9920419]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:37:53.191107 28403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef08a9e7e974b229df7b30cdb0de23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:53.191130 28012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef08a9e7e974b229df7b30cdb0de23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:53.192435 28532 tablet_bootstrap.cc:492] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: Bootstrap starting.
I20260806 22:37:53.192535 28531 tablet_bootstrap.cc:492] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: Bootstrap starting.
I20260806 22:37:53.192551 28206 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef08a9e7e974b229df7b30cdb0de23f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=54b193c0f234417593bf3b8fd9920419]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:37:53.192706 28206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef08a9e7e974b229df7b30cdb0de23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:53.192889 28532 tablet_bootstrap.cc:654] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:53.192996 28531 tablet_bootstrap.cc:654] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:53.193600 28532 tablet_bootstrap.cc:492] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: No bootstrap required, opened a new log
I20260806 22:37:53.193665 28532 ts_tablet_manager.cc:1397] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:53.193802 28532 raft_consensus.cc:348] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.193881 28532 raft_consensus.cc:374] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:53.193902 28532 raft_consensus.cc:729] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5abd7ffee617475c90484af02b2ebf79, State: Initialized, Role: FOLLOWER
I20260806 22:37:53.193979 28532 consensus_queue.cc:260] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.194083 28532 ts_tablet_manager.cc:1428] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:53.194224 28536 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.194310 28536 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:53.194355 28535 tablet_bootstrap.cc:492] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: Bootstrap starting.
I20260806 22:37:53.194785 28531 tablet_bootstrap.cc:492] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: No bootstrap required, opened a new log
I20260806 22:37:53.194782 28535 tablet_bootstrap.cc:654] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:53.194839 28531 ts_tablet_manager.cc:1397] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:53.194994 28531 raft_consensus.cc:348] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.195075 28531 raft_consensus.cc:374] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:53.195097 28531 raft_consensus.cc:729] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c8bba496cf467ead48e8f745353554, State: Initialized, Role: FOLLOWER
I20260806 22:37:53.195150 28531 consensus_queue.cc:260] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.195257 28531 ts_tablet_manager.cc:1428] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:53.195299 28539 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.195392 28539 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:53.195683 28535 tablet_bootstrap.cc:492] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: No bootstrap required, opened a new log
I20260806 22:37:53.195735 28535 ts_tablet_manager.cc:1397] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:53.195868 28535 raft_consensus.cc:348] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.195932 28535 raft_consensus.cc:374] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:53.195946 28535 raft_consensus.cc:729] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f476e38030d490f9c62b34e1d69460f, State: Initialized, Role: FOLLOWER
I20260806 22:37:53.196015 28535 consensus_queue.cc:260] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.196126 28535 ts_tablet_manager.cc:1428] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:53.196152 28543 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.196245 28543 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:53.451756 28536 raft_consensus.cc:482] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:53.451926 28536 raft_consensus.cc:504] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.452343 28536 leader_election.cc:288] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951), f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391)
I20260806 22:37:53.456696 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.452670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:48141 (local address 127.16.235.66:44951)
0806 22:37:53.452724 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.452727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.452835 (+ 108us) server_negotiation.cc:407] Connection header received
0806 22:37:53.452882 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.452884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.452895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.452985 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:53.453205 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.453980 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.455680 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.456288 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.456381 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.456392 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.456581 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.456624 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:37:53.456700 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.452769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:58545 (local address 127.16.235.67:39391)
0806 22:37:53.452912 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.452914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.452922 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:53.452947 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.452947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.452959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.453030 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:37:53.453195 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.453991 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.455681 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.456323 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.456392 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.456401 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.456581 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.456628 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260806 22:37:53.456724 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.452600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:44951 (local address 127.16.235.65:48141)
0806 22:37:53.452744 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.452757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.452825 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.452999 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.453000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.453021 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:53.453111 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.453113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.454082 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.454084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.455571 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.455573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.456399 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.456400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.456483 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.456490 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.456555 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.456610 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":151,"thread_start_us":71,"threads_started":1}
I20260806 22:37:53.456735 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.452600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:39391 (local address 127.16.235.65:58545)
0806 22:37:53.452757 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.452766 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.452825 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.453045 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.453046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.453048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:37:53.453117 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.453118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.454100 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.454101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.455560 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.455565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.456415 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.456416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.456483 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.456492 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.456555 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.456610 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":103,"thread_start_us":67,"threads_started":1}
I20260806 22:37:53.456998 28234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef08a9e7e974b229df7b30cdb0de23f" candidate_uuid: "5abd7ffee617475c90484af02b2ebf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f476e38030d490f9c62b34e1d69460f" is_pre_election: true
I20260806 22:37:53.457113 28234 raft_consensus.cc:2454] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5abd7ffee617475c90484af02b2ebf79 in term 0.
I20260806 22:37:53.457105 28431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef08a9e7e974b229df7b30cdb0de23f" candidate_uuid: "5abd7ffee617475c90484af02b2ebf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c8bba496cf467ead48e8f745353554" is_pre_election: true
I20260806 22:37:53.457197 28431 raft_consensus.cc:2454] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5abd7ffee617475c90484af02b2ebf79 in term 0.
I20260806 22:37:53.457320 27943 leader_election.cc:302] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f476e38030d490f9c62b34e1d69460f, 5abd7ffee617475c90484af02b2ebf79; no voters:
I20260806 22:37:53.457509 28536 raft_consensus.cc:2781] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:53.457561 28536 raft_consensus.cc:482] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:53.457583 28536 raft_consensus.cc:3037] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.458359 28536 raft_consensus.cc:504] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.458519 28536 leader_election.cc:288] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [CANDIDATE]: Term 1 election: Requested vote from peers 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951), f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391)
I20260806 22:37:53.458695 28234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef08a9e7e974b229df7b30cdb0de23f" candidate_uuid: "5abd7ffee617475c90484af02b2ebf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f476e38030d490f9c62b34e1d69460f"
I20260806 22:37:53.458720 28431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef08a9e7e974b229df7b30cdb0de23f" candidate_uuid: "5abd7ffee617475c90484af02b2ebf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c8bba496cf467ead48e8f745353554"
I20260806 22:37:53.458797 28431 raft_consensus.cc:3037] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.458801 28234 raft_consensus.cc:3037] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.459421 28431 raft_consensus.cc:2454] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5abd7ffee617475c90484af02b2ebf79 in term 1.
I20260806 22:37:53.459539 28234 raft_consensus.cc:2454] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5abd7ffee617475c90484af02b2ebf79 in term 1.
I20260806 22:37:53.459661 27941 leader_election.cc:302] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5abd7ffee617475c90484af02b2ebf79, f6c8bba496cf467ead48e8f745353554; no voters:
I20260806 22:37:53.459822 28536 raft_consensus.cc:2781] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:53.459985 28536 raft_consensus.cc:686] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [term 1 LEADER]: Becoming Leader. State: Replica: 5abd7ffee617475c90484af02b2ebf79, State: Running, Role: LEADER
I20260806 22:37:53.460079 28536 consensus_queue.cc:237] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.460494 28595 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: TxnStatusTablet state changed. Reason: New leader 5abd7ffee617475c90484af02b2ebf79. Latest consensus state: current_term: 1 leader_uuid: "5abd7ffee617475c90484af02b2ebf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.460618 28595 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:37:53.460814 28597 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:37:53.460958 28597 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:37:53.460911 27815 catalog_manager.cc:5495] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65). New cstate: current_term: 1 leader_uuid: "5abd7ffee617475c90484af02b2ebf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:53.481827 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.478931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47160 (local address 127.16.235.65:34241)
0806 22:37:53.478995 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.478997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.479010 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:37:53.479095 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.479096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.479115 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.479173 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:53.479303 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.479798 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.480942 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.481137 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.481191 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.481473 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.481580 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.481649 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.481737 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.481760 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:37:53.481866 28584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.478831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:34241 (local address 127.0.0.1:47160)
0806 22:37:53.478927 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.478943 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.479002 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.479180 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.479181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.479182 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:53.479244 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.479246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.479881 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.479882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.480846 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.480849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.481224 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.481226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.481301 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.481321 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.481663 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.481673 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.481727 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.481783 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 22:37:53.483349 28431 raft_consensus.cc:1261] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Refusing update from remote peer 5abd7ffee617475c90484af02b2ebf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:53.483445 28234 raft_consensus.cc:1261] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Refusing update from remote peer 5abd7ffee617475c90484af02b2ebf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:53.483655 28536 consensus_queue.cc:1035] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:53.483807 28595 consensus_queue.cc:1035] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:53.484534 28543 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: TxnStatusTablet state changed. Reason: New leader 5abd7ffee617475c90484af02b2ebf79. Latest consensus state: current_term: 1 leader_uuid: "5abd7ffee617475c90484af02b2ebf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.484642 28543 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:53.485018 28539 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: TxnStatusTablet state changed. Reason: New leader 5abd7ffee617475c90484af02b2ebf79. Latest consensus state: current_term: 1 leader_uuid: "5abd7ffee617475c90484af02b2ebf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.485116 28539 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:53.485410 28536 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5abd7ffee617475c90484af02b2ebf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.485518 28536 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:37:53.485880 28536 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5abd7ffee617475c90484af02b2ebf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.485972 28536 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P 5abd7ffee617475c90484af02b2ebf79: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:37:53.486048 28543 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5abd7ffee617475c90484af02b2ebf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.486138 28543 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P 2f476e38030d490f9c62b34e1d69460f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:53.486269 28605 mvcc.cc:205] Tried to move back new op lower bound from 7315684857785298944 to 7315684857693257728. Current Snapshot: MvccSnapshot[applied={T|T < 7315684857785298944}]
I20260806 22:37:53.486390 28539 tablet_replica.cc:439] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5abd7ffee617475c90484af02b2ebf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } } }
I20260806 22:37:53.486488 28539 tablet_replica.cc:442] T 1ef08a9e7e974b229df7b30cdb0de23f P f6c8bba496cf467ead48e8f745353554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:37:53.486831 28603 mvcc.cc:205] Tried to move back new op lower bound from 7315684857785298944 to 7315684857693257728. Current Snapshot: MvccSnapshot[applied={T|T < 7315684857785298944}]
I20260806 22:37:53.491724 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.488731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.16.235.67:39391)
0806 22:37:53.488771 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.488773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.488912 (+ 139us) server_negotiation.cc:407] Connection header received
0806 22:37:53.488981 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.488982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.488999 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.489082 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:53.489265 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.489796 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.490940 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.491130 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.491193 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.491368 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.491468 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.491548 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.491575 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.491599 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:37:53.497368 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.494604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:54859 (local address 127.16.235.65:34241)
0806 22:37:53.494664 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.494666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.494803 (+ 137us) server_negotiation.cc:407] Connection header received
0806 22:37:53.494882 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.494883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.494891 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.494938 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:53.495087 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.495550 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.496687 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.496902 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.496957 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.497163 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.497189 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.497227 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.497282 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.497308 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:37:53.497418 28617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.494525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:34241 (local address 127.16.235.67:54859)
0806 22:37:53.494743 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.494756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.494820 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.494942 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.494943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.494944 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:53.495021 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.495022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.495625 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.495626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.496606 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.496609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.496989 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.496990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.497057 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.497076 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.497229 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.497230 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.497280 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.497326 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":95,"threads_started":1}
I20260806 22:37:53.500193 28234 raft_consensus.cc:1261] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Refusing update from remote peer f6c8bba496cf467ead48e8f745353554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:53.500240 28037 raft_consensus.cc:1261] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Refusing update from remote peer f6c8bba496cf467ead48e8f745353554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:53.500500 28574 consensus_queue.cc:1035] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:53.500644 28539 consensus_queue.cc:1035] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:53.510234 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.507219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:42513 (local address 127.16.235.67:39391)
0806 22:37:53.507261 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.507263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.507426 (+ 163us) server_negotiation.cc:407] Connection header received
0806 22:37:53.507476 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.507477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.507488 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.507548 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:53.507749 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.508221 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.509540 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.509755 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.509811 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.510037 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.510064 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.510102 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.510158 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.510177 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:37:53.510509 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.507166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:39391 (local address 127.16.235.65:42513)
0806 22:37:53.507348 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.507360 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.507423 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.507600 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.507601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.507603 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:53.507674 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.507676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.508303 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.508304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.509425 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.509429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.509856 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.509857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.509928 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.509947 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.510103 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.510105 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.510154 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.510202 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260806 22:37:53.520517 27815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:53.520659 27815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:53.521384 27815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: Sending DeleteTablet for 3 replicas of tablet ebb66f3ae60c42f8be2b6f406a161d9a
I20260806 22:37:53.521690 28012 tablet_service.cc:1514] Processing DeleteTablet for tablet ebb66f3ae60c42f8be2b6f406a161d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:53 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47146
I20260806 22:37:53.521698 28206 tablet_service.cc:1514] Processing DeleteTablet for tablet ebb66f3ae60c42f8be2b6f406a161d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:53 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:56678
I20260806 22:37:53.521788 28403 tablet_service.cc:1514] Processing DeleteTablet for tablet ebb66f3ae60c42f8be2b6f406a161d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:53 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:38390
I20260806 22:37:53.522387 28629 tablet_replica.cc:330] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: stopping tablet replica
I20260806 22:37:53.522464 28629 raft_consensus.cc:2229] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:53.522512 28629 raft_consensus.cc:2258] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:53.522634 28632 tablet_replica.cc:330] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: stopping tablet replica
I20260806 22:37:53.522732 28632 raft_consensus.cc:2229] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:53.522770 28632 raft_consensus.cc:2258] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:53.523002 28629 ts_tablet_manager.cc:1905] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:53.524609 28632 ts_tablet_manager.cc:1905] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:53.525115 28629 ts_tablet_manager.cc:1918] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:53.525174 28629 log.cc:1196] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/ebb66f3ae60c42f8be2b6f406a161d9a
I20260806 22:37:53.525410 28629 ts_tablet_manager.cc:1939] T ebb66f3ae60c42f8be2b6f406a161d9a P 2f476e38030d490f9c62b34e1d69460f: Deleting consensus metadata
I20260806 22:37:53.525755 27791 catalog_manager.cc:4841] TS 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951): tablet ebb66f3ae60c42f8be2b6f406a161d9a (table test-table [id=5d3b0a995ccc4f609b7490558c42afce]) successfully deleted
I20260806 22:37:53.526748 28632 ts_tablet_manager.cc:1918] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:53.526821 28632 log.cc:1196] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/ebb66f3ae60c42f8be2b6f406a161d9a
I20260806 22:37:53.527145 28632 ts_tablet_manager.cc:1939] T ebb66f3ae60c42f8be2b6f406a161d9a P 5abd7ffee617475c90484af02b2ebf79: Deleting consensus metadata
I20260806 22:37:53.527506 27787 catalog_manager.cc:4841] TS 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241): tablet ebb66f3ae60c42f8be2b6f406a161d9a (table test-table [id=5d3b0a995ccc4f609b7490558c42afce]) successfully deleted
I20260806 22:37:53.530843 28633 tablet_replica.cc:330] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: stopping tablet replica
I20260806 22:37:53.530942 28633 raft_consensus.cc:2229] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:53.531064 28633 raft_consensus.cc:2258] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:53.531394 28633 ts_tablet_manager.cc:1905] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:53.532838 28633 ts_tablet_manager.cc:1918] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:53.532919 28633 log.cc:1196] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/ebb66f3ae60c42f8be2b6f406a161d9a
I20260806 22:37:53.533207 28633 ts_tablet_manager.cc:1939] T ebb66f3ae60c42f8be2b6f406a161d9a P f6c8bba496cf467ead48e8f745353554: Deleting consensus metadata
I20260806 22:37:53.533573 27790 catalog_manager.cc:4841] TS f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391): tablet ebb66f3ae60c42f8be2b6f406a161d9a (table test-table [id=5d3b0a995ccc4f609b7490558c42afce]) successfully deleted
Aug 06 22:37:53 dist-test-slave-3kk6 krb5kdc[27738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:37:53 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:53 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:37:53.559965 28585 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:53.551817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48756 (local address 127.16.235.126:43797)
0806 22:37:53.551873 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.551876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.552233 (+ 357us) server_negotiation.cc:407] Connection header received
0806 22:37:53.552315 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.552316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.552334 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.552411 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:53.553201 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.553986 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.555167 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.555470 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.555556 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.559893 (+ 4337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48756: BlockingRecv error: recv got EOF from 127.0.0.1:48756 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 22:37:53 dist-test-slave-3kk6 krb5kdc[27738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM
I20260806 22:37:53.571445 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.561828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.16.235.126:43797)
0806 22:37:53.561886 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.561888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.561989 (+ 101us) server_negotiation.cc:407] Connection header received
0806 22:37:53.562045 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.562046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.562058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.562133 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:53.562796 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.563552 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.564623 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.564931 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.565018 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.568019 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:53.568021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:53.568022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:53.568030 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:53.569907 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:53.570365 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:53.570366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:53.570367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:53.570435 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:53.570702 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:53.570703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:53.570703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:53.570849 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:53.570960 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.571133 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.571233 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:37:53.573622 27815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:37:53.573812 27815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:53.575434 28206 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb4ebd4be424626b2259db99859c2d5 (DEFAULT_TABLE table=test-table [id=0eca2a6d9f014972b72834e016a8e1f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:53.575457 28403 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb4ebd4be424626b2259db99859c2d5 (DEFAULT_TABLE table=test-table [id=0eca2a6d9f014972b72834e016a8e1f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:53.575595 28206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb4ebd4be424626b2259db99859c2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:53.575610 28403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb4ebd4be424626b2259db99859c2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:53.576979 28535 tablet_bootstrap.cc:492] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: Bootstrap starting.
I20260806 22:37:53.577117 28531 tablet_bootstrap.cc:492] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: Bootstrap starting.
I20260806 22:37:53.577337 28535 tablet_bootstrap.cc:654] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:53.577569 28531 tablet_bootstrap.cc:654] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:53.578112 28012 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb4ebd4be424626b2259db99859c2d5 (DEFAULT_TABLE table=test-table [id=0eca2a6d9f014972b72834e016a8e1f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:53.578260 28012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb4ebd4be424626b2259db99859c2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:53.579604 28532 tablet_bootstrap.cc:492] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: Bootstrap starting.
I20260806 22:37:53.580021 28532 tablet_bootstrap.cc:654] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:53.580729 28532 tablet_bootstrap.cc:492] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: No bootstrap required, opened a new log
I20260806 22:37:53.580803 28532 ts_tablet_manager.cc:1397] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:53.580976 28532 raft_consensus.cc:348] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.581067 28532 raft_consensus.cc:374] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:53.581089 28532 raft_consensus.cc:729] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5abd7ffee617475c90484af02b2ebf79, State: Initialized, Role: FOLLOWER
I20260806 22:37:53.581084 28535 tablet_bootstrap.cc:492] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: No bootstrap required, opened a new log
I20260806 22:37:53.581135 28535 ts_tablet_manager.cc:1397] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 22:37:53.581135 28532 consensus_queue.cc:260] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.581231 28532 ts_tablet_manager.cc:1428] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:53.581260 28535 raft_consensus.cc:348] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.581331 28535 raft_consensus.cc:374] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:53.581350 28535 raft_consensus.cc:729] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f476e38030d490f9c62b34e1d69460f, State: Initialized, Role: FOLLOWER
I20260806 22:37:53.581394 28535 consensus_queue.cc:260] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.581476 28535 ts_tablet_manager.cc:1428] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:53.582928 28531 tablet_bootstrap.cc:492] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: No bootstrap required, opened a new log
I20260806 22:37:53.582998 28531 ts_tablet_manager.cc:1397] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260806 22:37:53.583153 28531 raft_consensus.cc:348] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.583238 28531 raft_consensus.cc:374] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:53.583259 28531 raft_consensus.cc:729] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c8bba496cf467ead48e8f745353554, State: Initialized, Role: FOLLOWER
I20260806 22:37:53.583315 28531 consensus_queue.cc:260] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.583410 28531 ts_tablet_manager.cc:1428] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:37:53.686640 28539 raft_consensus.cc:482] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:53.686800 28539 raft_consensus.cc:504] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.686985 28539 leader_election.cc:288] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241), 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951)
I20260806 22:37:53.687270 28234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb4ebd4be424626b2259db99859c2d5" candidate_uuid: "f6c8bba496cf467ead48e8f745353554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f476e38030d490f9c62b34e1d69460f" is_pre_election: true
I20260806 22:37:53.687288 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb4ebd4be424626b2259db99859c2d5" candidate_uuid: "f6c8bba496cf467ead48e8f745353554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5abd7ffee617475c90484af02b2ebf79" is_pre_election: true
I20260806 22:37:53.687393 28234 raft_consensus.cc:2454] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6c8bba496cf467ead48e8f745353554 in term 0.
I20260806 22:37:53.687412 28037 raft_consensus.cc:2454] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6c8bba496cf467ead48e8f745353554 in term 0.
I20260806 22:37:53.687597 28341 leader_election.cc:302] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f476e38030d490f9c62b34e1d69460f, f6c8bba496cf467ead48e8f745353554; no voters:
I20260806 22:37:53.687738 28539 raft_consensus.cc:2781] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:53.687780 28539 raft_consensus.cc:482] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:53.687800 28539 raft_consensus.cc:3037] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.688778 28539 raft_consensus.cc:504] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.688927 28539 leader_election.cc:288] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [CANDIDATE]: Term 1 election: Requested vote from peers 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241), 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951)
I20260806 22:37:53.689134 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb4ebd4be424626b2259db99859c2d5" candidate_uuid: "f6c8bba496cf467ead48e8f745353554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5abd7ffee617475c90484af02b2ebf79"
I20260806 22:37:53.689116 28234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb4ebd4be424626b2259db99859c2d5" candidate_uuid: "f6c8bba496cf467ead48e8f745353554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f476e38030d490f9c62b34e1d69460f"
I20260806 22:37:53.689231 28234 raft_consensus.cc:3037] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.689234 28037 raft_consensus.cc:3037] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.690027 28037 raft_consensus.cc:2454] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6c8bba496cf467ead48e8f745353554 in term 1.
I20260806 22:37:53.690040 28234 raft_consensus.cc:2454] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6c8bba496cf467ead48e8f745353554 in term 1.
I20260806 22:37:53.690228 28341 leader_election.cc:302] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f476e38030d490f9c62b34e1d69460f, f6c8bba496cf467ead48e8f745353554; no voters:
I20260806 22:37:53.690354 28539 raft_consensus.cc:2781] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:53.690428 28539 raft_consensus.cc:686] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 1 LEADER]: Becoming Leader. State: Replica: f6c8bba496cf467ead48e8f745353554, State: Running, Role: LEADER
I20260806 22:37:53.690474 28539 consensus_queue.cc:237] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.691144 27815 catalog_manager.cc:5495] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6c8bba496cf467ead48e8f745353554 (127.16.235.67). New cstate: current_term: 1 leader_uuid: "f6c8bba496cf467ead48e8f745353554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } health_report { overall_health: HEALTHY } } }
I20260806 22:37:53.705607 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.703244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.16.235.67:39391)
0806 22:37:53.703291 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.703294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.703304 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:37:53.703355 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.703356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.703368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.703423 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:53.703582 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.704082 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.704903 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.705092 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.705173 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.705336 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.705377 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.705439 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.705502 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.705521 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:37:53.706512 28037 raft_consensus.cc:1261] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Refusing update from remote peer f6c8bba496cf467ead48e8f745353554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:53.706533 28234 raft_consensus.cc:1261] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Refusing update from remote peer f6c8bba496cf467ead48e8f745353554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:53.706771 28539 consensus_queue.cc:1035] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:53.706924 28621 consensus_queue.cc:1035] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:53.709319 27815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:53.709417 27815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:53.710103 27815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: Sending DeleteTablet for 3 replicas of tablet 5fb4ebd4be424626b2259db99859c2d5
I20260806 22:37:53.710423 28206 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb4ebd4be424626b2259db99859c2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:53 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:56678
I20260806 22:37:53.710536 28629 tablet_replica.cc:330] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: stopping tablet replica
I20260806 22:37:53.710542 28012 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb4ebd4be424626b2259db99859c2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:53 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47146
I20260806 22:37:53.710608 28629 raft_consensus.cc:2229] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:53.710628 28632 tablet_replica.cc:330] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: stopping tablet replica
I20260806 22:37:53.710630 28403 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb4ebd4be424626b2259db99859c2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:53 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:38390
I20260806 22:37:53.710708 28632 raft_consensus.cc:2229] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:53.710721 28629 raft_consensus.cc:2258] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:53.710758 28632 raft_consensus.cc:2258] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:53.710767 28633 tablet_replica.cc:330] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: stopping tablet replica
I20260806 22:37:53.710834 28633 raft_consensus.cc:2229] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:53.710901 28633 raft_consensus.cc:2258] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:53.711202 28629 ts_tablet_manager.cc:1905] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:53.711201 27815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 22:37:53.711287 28632 ts_tablet_manager.cc:1905] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 22:37:53.711354 27815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:37:53.712972 28632 ts_tablet_manager.cc:1918] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:53.713028 28632 log.cc:1196] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/5fb4ebd4be424626b2259db99859c2d5
I20260806 22:37:53.712993 28629 ts_tablet_manager.cc:1918] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:53.713074 28629 log.cc:1196] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/5fb4ebd4be424626b2259db99859c2d5
I20260806 22:37:53.713205 28632 ts_tablet_manager.cc:1939] T 5fb4ebd4be424626b2259db99859c2d5 P 5abd7ffee617475c90484af02b2ebf79: Deleting consensus metadata
I20260806 22:37:53.713338 28629 ts_tablet_manager.cc:1939] T 5fb4ebd4be424626b2259db99859c2d5 P 2f476e38030d490f9c62b34e1d69460f: Deleting consensus metadata
I20260806 22:37:53.713362 28403 tablet_service.cc:1467] Processing CreateTablet for tablet dee950e81dc74b1f90baf089dd3d9880 (DEFAULT_TABLE table=test-table [id=4645be26dde74f88bd6d906be64523a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:53.713471 28403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dee950e81dc74b1f90baf089dd3d9880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:53.713527 28206 tablet_service.cc:1467] Processing CreateTablet for tablet dee950e81dc74b1f90baf089dd3d9880 (DEFAULT_TABLE table=test-table [id=4645be26dde74f88bd6d906be64523a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:53.713624 28206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dee950e81dc74b1f90baf089dd3d9880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:53.713809 27791 catalog_manager.cc:4841] TS 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951): tablet 5fb4ebd4be424626b2259db99859c2d5 (table test-table [id=0eca2a6d9f014972b72834e016a8e1f7]) successfully deleted
I20260806 22:37:53.714509 28531 tablet_bootstrap.cc:492] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: Bootstrap starting.
I20260806 22:37:53.714569 28535 tablet_bootstrap.cc:492] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: Bootstrap starting.
I20260806 22:37:53.714833 27787 catalog_manager.cc:4841] TS 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241): tablet 5fb4ebd4be424626b2259db99859c2d5 (table test-table [id=0eca2a6d9f014972b72834e016a8e1f7]) successfully deleted
I20260806 22:37:53.714921 28531 tablet_bootstrap.cc:654] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:53.714917 28535 tablet_bootstrap.cc:654] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:53.715013 28012 tablet_service.cc:1467] Processing CreateTablet for tablet dee950e81dc74b1f90baf089dd3d9880 (DEFAULT_TABLE table=test-table [id=4645be26dde74f88bd6d906be64523a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:37:53.715117 28012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dee950e81dc74b1f90baf089dd3d9880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:53.715420 28633 ts_tablet_manager.cc:1905] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:53.715731 28535 tablet_bootstrap.cc:492] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: No bootstrap required, opened a new log
I20260806 22:37:53.715786 28535 ts_tablet_manager.cc:1397] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:53.715898 28535 raft_consensus.cc:348] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.715962 28535 raft_consensus.cc:374] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:53.715981 28535 raft_consensus.cc:729] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f476e38030d490f9c62b34e1d69460f, State: Initialized, Role: FOLLOWER
I20260806 22:37:53.716029 28535 consensus_queue.cc:260] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.716115 28535 ts_tablet_manager.cc:1428] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:53.716467 28532 tablet_bootstrap.cc:492] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79: Bootstrap starting.
I20260806 22:37:53.716810 28532 tablet_bootstrap.cc:654] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:53.717059 28633 ts_tablet_manager.cc:1918] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:37:53.717116 28633 log.cc:1196] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/5fb4ebd4be424626b2259db99859c2d5
I20260806 22:37:53.717382 28633 ts_tablet_manager.cc:1939] T 5fb4ebd4be424626b2259db99859c2d5 P f6c8bba496cf467ead48e8f745353554: Deleting consensus metadata
I20260806 22:37:53.717689 27790 catalog_manager.cc:4841] TS f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391): tablet 5fb4ebd4be424626b2259db99859c2d5 (table test-table [id=0eca2a6d9f014972b72834e016a8e1f7]) successfully deleted
I20260806 22:37:53.717864 28531 tablet_bootstrap.cc:492] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: No bootstrap required, opened a new log
I20260806 22:37:53.717918 28531 ts_tablet_manager.cc:1397] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 22:37:53.718031 28531 raft_consensus.cc:348] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.718098 28531 raft_consensus.cc:374] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:53.718142 28531 raft_consensus.cc:729] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c8bba496cf467ead48e8f745353554, State: Initialized, Role: FOLLOWER
I20260806 22:37:53.718195 28531 consensus_queue.cc:260] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.718298 28531 ts_tablet_manager.cc:1428] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:53.718742 28532 tablet_bootstrap.cc:492] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79: No bootstrap required, opened a new log
I20260806 22:37:53.718796 28532 ts_tablet_manager.cc:1397] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:37:53.718912 28532 raft_consensus.cc:348] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.718986 28532 raft_consensus.cc:374] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:53.719002 28532 raft_consensus.cc:729] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5abd7ffee617475c90484af02b2ebf79, State: Initialized, Role: FOLLOWER
I20260806 22:37:53.719035 28532 consensus_queue.cc:260] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.719136 28532 ts_tablet_manager.cc:1428] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:53.733347 28543 raft_consensus.cc:482] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:37:53.733493 28543 raft_consensus.cc:504] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.733711 28543 leader_election.cc:288] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241), f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391)
I20260806 22:37:53.733968 28431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dee950e81dc74b1f90baf089dd3d9880" candidate_uuid: "2f476e38030d490f9c62b34e1d69460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c8bba496cf467ead48e8f745353554" is_pre_election: true
I20260806 22:37:53.734057 28431 raft_consensus.cc:2454] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f476e38030d490f9c62b34e1d69460f in term 0.
I20260806 22:37:53.733976 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dee950e81dc74b1f90baf089dd3d9880" candidate_uuid: "2f476e38030d490f9c62b34e1d69460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5abd7ffee617475c90484af02b2ebf79" is_pre_election: true
I20260806 22:37:53.734103 28037 raft_consensus.cc:2454] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f476e38030d490f9c62b34e1d69460f in term 0.
I20260806 22:37:53.734256 28138 leader_election.cc:302] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f476e38030d490f9c62b34e1d69460f, f6c8bba496cf467ead48e8f745353554; no voters:
I20260806 22:37:53.734375 28543 raft_consensus.cc:2781] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:37:53.734450 28543 raft_consensus.cc:482] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:37:53.734484 28543 raft_consensus.cc:3037] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.735347 28543 raft_consensus.cc:504] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.735509 28543 leader_election.cc:288] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [CANDIDATE]: Term 1 election: Requested vote from peers 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241), f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391)
I20260806 22:37:53.735685 28431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dee950e81dc74b1f90baf089dd3d9880" candidate_uuid: "2f476e38030d490f9c62b34e1d69460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c8bba496cf467ead48e8f745353554"
I20260806 22:37:53.735685 28037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dee950e81dc74b1f90baf089dd3d9880" candidate_uuid: "2f476e38030d490f9c62b34e1d69460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5abd7ffee617475c90484af02b2ebf79"
I20260806 22:37:53.735780 28037 raft_consensus.cc:3037] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.735780 28431 raft_consensus.cc:3037] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:53.736582 28431 raft_consensus.cc:2454] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f476e38030d490f9c62b34e1d69460f in term 1.
I20260806 22:37:53.736582 28037 raft_consensus.cc:2454] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f476e38030d490f9c62b34e1d69460f in term 1.
I20260806 22:37:53.736773 28135 leader_election.cc:302] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f476e38030d490f9c62b34e1d69460f, 5abd7ffee617475c90484af02b2ebf79; no voters:
I20260806 22:37:53.736881 28543 raft_consensus.cc:2781] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:53.737047 28543 raft_consensus.cc:686] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 1 LEADER]: Becoming Leader. State: Replica: 2f476e38030d490f9c62b34e1d69460f, State: Running, Role: LEADER
I20260806 22:37:53.737097 28543 consensus_queue.cc:237] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } }
I20260806 22:37:53.737761 27813 catalog_manager.cc:5495] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f476e38030d490f9c62b34e1d69460f (127.16.235.66). New cstate: current_term: 1 leader_uuid: "2f476e38030d490f9c62b34e1d69460f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f476e38030d490f9c62b34e1d69460f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 44951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20260806 22:37:53.749410 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.747447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56698 (local address 127.16.235.66:44951)
0806 22:37:53.747492 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.747494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.747509 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:37:53.747567 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.747567 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.747576 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.747620 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:53.747703 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.748175 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.748849 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.749027 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.749078 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.749183 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.749207 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.749240 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.749277 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.749292 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:37:53.754457 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.752011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:52209 (local address 127.16.235.65:34241)
0806 22:37:53.752080 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.752082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.752199 (+ 117us) server_negotiation.cc:407] Connection header received
0806 22:37:53.752292 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.752293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.752303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.752364 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:53.752484 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.752947 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.753722 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.753917 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.754003 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.754187 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.754225 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.754292 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.754356 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.754382 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:37:53.754521 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.751920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:34241 (local address 127.16.235.66:52209)
0806 22:37:53.752119 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.752131 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.752218 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.752368 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.752368 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.752370 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:53.752423 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.752425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.753030 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.753031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.753629 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.753630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.754007 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.754008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.754066 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.754090 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.754294 (+ 204us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.754297 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.754351 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.754421 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260806 22:37:53.756887 28037 raft_consensus.cc:1261] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Refusing update from remote peer 2f476e38030d490f9c62b34e1d69460f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:53.756887 28431 raft_consensus.cc:1261] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Refusing update from remote peer 2f476e38030d490f9c62b34e1d69460f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:37:53.757128 28543 consensus_queue.cc:1035] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5abd7ffee617475c90484af02b2ebf79" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:53.757264 28659 consensus_queue.cc:1035] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6c8bba496cf467ead48e8f745353554" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:37:53.764636 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.762369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:59067 (local address 127.16.235.66:44951)
0806 22:37:53.762415 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:53.762417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:53.762425 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:37:53.762445 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:53.762446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:53.762454 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:53.762511 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:37:53.762669 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.763229 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.763931 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.764111 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.764198 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.764362 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.764400 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.764460 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:53.764527 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 22:37:53.764550 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:37:53.764665 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.762273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:44951 (local address 127.16.235.65:59067)
0806 22:37:53.762312 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.762324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:53.762371 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:53.762515 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:53.762516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:53.762517 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:37:53.762569 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.762571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.763298 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.763299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.763842 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:53.763843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:53.764220 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:53.764221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:53.764285 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:53.764294 (+ 9us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:37:53.764470 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:37:53.764473 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:37:53.764525 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:37:53.764576 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 22:37:53.771878 27813 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:53.772038 27813 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:37:53.773180 27813 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 011fee1e76ab400a86ad8ac2b66dccf5: Sending DeleteTablet for 3 replicas of tablet dee950e81dc74b1f90baf089dd3d9880
I20260806 22:37:53.773506 28403 tablet_service.cc:1514] Processing DeleteTablet for tablet dee950e81dc74b1f90baf089dd3d9880 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:53 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:38390
I20260806 22:37:53.773609 28633 tablet_replica.cc:330] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: stopping tablet replica
I20260806 22:37:53.773664 28633 raft_consensus.cc:2229] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:53.773705 28633 raft_consensus.cc:2258] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:53.773783 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 27918
I20260806 22:37:53.773885 28012 tablet_service.cc:1514] Processing DeleteTablet for tablet dee950e81dc74b1f90baf089dd3d9880 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:53 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47146
I20260806 22:37:53.773976 28632 tablet_replica.cc:330] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79: stopping tablet replica
I20260806 22:37:53.774024 28632 raft_consensus.cc:2229] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:37:53.774057 28632 raft_consensus.cc:2258] T dee950e81dc74b1f90baf089dd3d9880 P 5abd7ffee617475c90484af02b2ebf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:53.778982 28206 tablet_service.cc:1514] Processing DeleteTablet for tablet dee950e81dc74b1f90baf089dd3d9880 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:37:53 UTC) from {username='oryx', principal='oryx/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:56678
I20260806 22:37:53.779126 28629 tablet_replica.cc:330] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: stopping tablet replica
I20260806 22:37:53.779194 28629 raft_consensus.cc:2229] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:37:53.779284 28629 raft_consensus.cc:2258] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:37:53.779968 28629 ts_tablet_manager.cc:1905] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:53.781282 28633 ts_tablet_manager.cc:1905] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:37:53.781774 28629 ts_tablet_manager.cc:1918] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:53.781839 28629 log.cc:1196] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/dee950e81dc74b1f90baf089dd3d9880
I20260806 22:37:53.782073 28629 ts_tablet_manager.cc:1939] T dee950e81dc74b1f90baf089dd3d9880 P 2f476e38030d490f9c62b34e1d69460f: Deleting consensus metadata
I20260806 22:37:53.782536 27791 catalog_manager.cc:4841] TS 2f476e38030d490f9c62b34e1d69460f (127.16.235.66:44951): tablet dee950e81dc74b1f90baf089dd3d9880 (table test-table [id=4645be26dde74f88bd6d906be64523a7]) successfully deleted
I20260806 22:37:53.782959 28633 ts_tablet_manager.cc:1918] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:37:53.783013 28633 log.cc:1196] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/dee950e81dc74b1f90baf089dd3d9880
I20260806 22:37:53.783241 28633 ts_tablet_manager.cc:1939] T dee950e81dc74b1f90baf089dd3d9880 P f6c8bba496cf467ead48e8f745353554: Deleting consensus metadata
I20260806 22:37:53.783617 27790 catalog_manager.cc:4841] TS f6c8bba496cf467ead48e8f745353554 (127.16.235.67:39391): tablet dee950e81dc74b1f90baf089dd3d9880 (table test-table [id=4645be26dde74f88bd6d906be64523a7]) successfully deleted
W20260806 22:37:53.788324 27787 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:34241 (error 108)
I20260806 22:37:53.788659 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 28113
I20260806 22:37:53.794067 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:53.788746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:34241 (local address 127.0.0.1:47170)
0806 22:37:53.793937 (+ 5191us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:53.794025 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5165,"thread_start_us":69,"threads_started":1}
W20260806 22:37:53.794220 27787 catalog_manager.cc:4584] TS 5abd7ffee617475c90484af02b2ebf79 (127.16.235.65:34241): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dee950e81dc74b1f90baf089dd3d9880: Network error: Client connection negotiation failed: client connection to 127.16.235.65:34241: connect: Connection refused (error 111)
I20260806 22:37:53.802985 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 28312
I20260806 22:37:53.818022 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 27757
2026-08-06T22:37:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3324 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:53 dist-test-slave-3kk6 krb5kdc[28675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:53 dist-test-slave-3kk6 krb5kdc[28675](info): set up 2 sockets
Aug 06 22:37:53 dist-test-slave-3kk6 krb5kdc[28675](info): commencing operation
krb5kdc: starting...
W20260806 22:37:55.076108 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:55 dist-test-slave-3kk6 krb5kdc[28675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:55.205395 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:41681
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:44169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:41681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:55.291392 28689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:55.291625 28689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:55.291649 28689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:55.292917 28689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:55.292966 28689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:55.292981 28689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:55.293027 28689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:55.293046 28689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:55.294584 28689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:44169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:41681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:41681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:55.294889 28689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:55.295148 28689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:55.300189 28698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:55.300242 28700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:55.300203 28697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:55.300463 28689 server_base.cc:1034] running on GCE node
I20260806 22:37:55.300683 28689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:55.301002 28689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:55.302186 28689 hybrid_clock.cc:648] HybridClock initialized: now 1786055875302156 us; error 46 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T22:37:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1489 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:37:55 dist-test-slave-3kk6 krb5kdc[28705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:37:55 dist-test-slave-3kk6 krb5kdc[28705](info): set up 2 sockets
Aug 06 22:37:55 dist-test-slave-3kk6 krb5kdc[28705](info): commencing operation
krb5kdc: starting...
W20260806 22:37:56.542830 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:37:56 dist-test-slave-3kk6 krb5kdc[28705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:37:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:37:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:37:56.673827 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37773
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:37773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:56.759331 28724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:56.759608 28724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:56.759646 28724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:56.760872 28724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:56.760953 28724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:56.760978 28724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:56.760998 28724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:56.761029 28724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:56.762598 28724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:37773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:56.762929 28724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:56.763187 28724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:56.768261 28738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:56.768263 28735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:56.768481 28736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:56.768406 28724 server_base.cc:1034] running on GCE node
I20260806 22:37:56.768776 28724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:56.769044 28724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:56.770192 28724 hybrid_clock.cc:648] HybridClock initialized: now 1786055876770178 us; error 42 us; skew 500 ppm
Aug 06 22:37:56 dist-test-slave-3kk6 krb5kdc[28705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055876, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:56.772904 28724 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:56.773442 28724 webserver.cc:460] Webserver started at http://127.16.235.126:36739/ using document root <none> and password file <none>
I20260806 22:37:56.773691 28724 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:56.773765 28724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:56.773929 28724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:56.775148 28724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "0dd49bb3aeee4d36ba12eb48cc190009"
format_stamp: "Formatted at 2026-08-06 22:37:56 on dist-test-slave-3kk6"
server_key: "87aed152c4f589c37fa120ac62f36fb9"
server_key_iv: "e6bc812727c8007787909f51c087d18d"
server_key_version: "encryptionkey@0"
I20260806 22:37:56.775537 28724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "0dd49bb3aeee4d36ba12eb48cc190009"
format_stamp: "Formatted at 2026-08-06 22:37:56 on dist-test-slave-3kk6"
server_key: "87aed152c4f589c37fa120ac62f36fb9"
server_key_iv: "e6bc812727c8007787909f51c087d18d"
server_key_version: "encryptionkey@0"
I20260806 22:37:56.776973 28724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:56.777794 28748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:56.777994 28724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:56.778108 28724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "0dd49bb3aeee4d36ba12eb48cc190009"
format_stamp: "Formatted at 2026-08-06 22:37:56 on dist-test-slave-3kk6"
server_key: "87aed152c4f589c37fa120ac62f36fb9"
server_key_iv: "e6bc812727c8007787909f51c087d18d"
server_key_version: "encryptionkey@0"
I20260806 22:37:56.778187 28724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:56.788118 28724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:56.790062 28724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:56.790237 28724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:56.794838 28724 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:37773
I20260806 22:37:56.794875 28814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:37773 every 8 connection(s)
I20260806 22:37:56.795260 28724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:56.796162 28815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:37:56.798601 28815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: Bootstrap starting.
I20260806 22:37:56.799288 28815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: Neither blocks nor log segments found. Creating new log.
I20260806 22:37:56.799639 28815 log.cc:824] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:56.799762 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 28724
I20260806 22:37:56.799881 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:37:56.800104 17325 external_mini_cluster.cc:1428] Setting key ad84fb78eedfa3e9558b0a8648d94593
I20260806 22:37:56.800560 28815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: No bootstrap required, opened a new log
I20260806 22:37:56.802788 28815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } }
I20260806 22:37:56.803009 28815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:37:56.803056 28815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dd49bb3aeee4d36ba12eb48cc190009, State: Initialized, Role: FOLLOWER
I20260806 22:37:56.803202 28815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } }
I20260806 22:37:56.803301 28815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:56.803334 28815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:56.803382 28815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:37:56.804164 28815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } }
I20260806 22:37:56.804279 28815 leader_election.cc:302] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dd49bb3aeee4d36ba12eb48cc190009; no voters:
I20260806 22:37:56.804438 28815 leader_election.cc:288] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:37:56.804504 28822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:37:56.804699 28815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:56.804713 28822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 1 LEADER]: Becoming Leader. State: Replica: 0dd49bb3aeee4d36ba12eb48cc190009, State: Running, Role: LEADER
I20260806 22:37:56.804931 28822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } }
I20260806 22:37:56.805296 28826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } } }
I20260806 22:37:56.805377 28826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:56.805578 28827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dd49bb3aeee4d36ba12eb48cc190009. Latest consensus state: current_term: 1 leader_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } } }
I20260806 22:37:56.805640 28827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: This master's current role is: LEADER
W20260806 22:37:56.806784 28844 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:37:56.806849 28844 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:37:56.806907 28836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:56.807097 28836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:37:56 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:56.809197 28836 catalog_manager.cc:1347] Generated new cluster ID: 2fb0a22cd33c4b4f8806daf4b8aee40f
I20260806 22:37:56.809252 28836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:56.813943 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:56.800667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.16.235.126:37773)
0806 22:37:56.800863 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:56.800873 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:56.800907 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:37:56.801927 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:56.801938 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:56.802003 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:56.802172 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:56.802865 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:56.804632 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:56.805844 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:56.806552 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:56.806693 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:56.809633 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:56.809638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:56.809638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:56.809654 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:56.811875 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:56.812283 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:56.812286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:56.812286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:56.812415 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:56.812627 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:56.812627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:56.812628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:56.812959 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:56.813505 (+ 546us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:56.813582 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:37:56.813730 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260806 22:37:56.831105 28836 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:37:56.831972 28836 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:56.848011 28836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: Generated new TSK 0
I20260806 22:37:56.848282 28836 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:37:56.878104 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37773
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:56.964699 28851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:56.964924 28851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:56.964949 28851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:56.966140 28851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:56.966189 28851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:56.966216 28851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:56.967789 28851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:56.968055 28851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:56.968312 28851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:56.973459 28856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:56.973459 28862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:56.973491 28857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:56.973750 28851 server_base.cc:1034] running on GCE node
I20260806 22:37:56.974083 28851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:56.974362 28851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:56.975544 28851 hybrid_clock.cc:648] HybridClock initialized: now 1786055876975528 us; error 32 us; skew 500 ppm
Aug 06 22:37:56 dist-test-slave-3kk6 krb5kdc[28705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055876, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:56.978229 28851 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:37:56.978775 28851 webserver.cc:460] Webserver started at http://127.16.235.65:43025/ using document root <none> and password file <none>
I20260806 22:37:56.979015 28851 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:56.979086 28851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:56.979252 28851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:56.980346 28851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "5ea7c46d18f64c2ab312fd567cd0f4ae"
format_stamp: "Formatted at 2026-08-06 22:37:56 on dist-test-slave-3kk6"
server_key: "88edf794e06f850421a2a65feb77f0b9"
server_key_iv: "ab8778a47a78af2758357d6b346750fc"
server_key_version: "encryptionkey@0"
I20260806 22:37:56.980751 28851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "5ea7c46d18f64c2ab312fd567cd0f4ae"
format_stamp: "Formatted at 2026-08-06 22:37:56 on dist-test-slave-3kk6"
server_key: "88edf794e06f850421a2a65feb77f0b9"
server_key_iv: "ab8778a47a78af2758357d6b346750fc"
server_key_version: "encryptionkey@0"
I20260806 22:37:56.982282 28851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:56.983238 28869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:56.983464 28851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:37:56.983582 28851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "5ea7c46d18f64c2ab312fd567cd0f4ae"
format_stamp: "Formatted at 2026-08-06 22:37:56 on dist-test-slave-3kk6"
server_key: "88edf794e06f850421a2a65feb77f0b9"
server_key_iv: "ab8778a47a78af2758357d6b346750fc"
server_key_version: "encryptionkey@0"
I20260806 22:37:56.983660 28851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:57.012072 28851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:57.014338 28851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:57.014540 28851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:57.014894 28851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:57.015275 28851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:57.015357 28851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.015434 28851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:57.015465 28851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.021708 28851 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:38177
I20260806 22:37:57.021762 29028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:38177 every 8 connection(s)
I20260806 22:37:57.022117 28851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:57.024992 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 28851
I20260806 22:37:57.025135 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:37:57.025352 17325 external_mini_cluster.cc:1428] Setting key a2c7ddbeca45af2e0b888c75c15dda93
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055876, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:57.032965 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.022947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:47167 (local address 127.16.235.126:37773)
0806 22:37:57.022996 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:57.022998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:57.023212 (+ 214us) server_negotiation.cc:407] Connection header received
0806 22:37:57.023758 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:57.023759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:57.023771 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:57.023839 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:57.024834 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.025497 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.026541 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.026857 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.026957 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.029617 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:57.029619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:57.029619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:57.029628 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:57.031371 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:57.031863 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:57.031864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:57.031864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:57.031936 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:57.032236 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:57.032237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:57.032237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:57.032372 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:57.032455 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:57.032758 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 22:37:57.032862 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:37:57.033236 29032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.022932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:47167)
0806 22:37:57.023095 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.023135 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:57.023637 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:57.023893 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:57.023897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:57.024508 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:57.024720 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.024724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.025608 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.025609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.026433 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.026435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.026973 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.026974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.027110 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.027736 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:57.027749 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:57.029502 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:57.031487 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:57.031495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:57.031498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:57.031767 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:57.032013 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:57.032014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:57.032015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:57.032136 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:57.032446 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:57.032454 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:57.032612 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 22:37:57.032734 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 22:37:57.032912 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260806 22:37:57.033771 29029 heartbeater.cc:347] Connected to a master server at 127.16.235.126:37773
I20260806 22:37:57.033903 29029 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:57.034137 29029 heartbeater.cc:511] Master 127.16.235.126:37773 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
I20260806 22:37:57.034806 28767 ts_manager.cc:194] Registered new tserver with Master: 5ea7c46d18f64c2ab312fd567cd0f4ae (127.16.235.65:38177)
I20260806 22:37:57.035494 28767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:47167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:37:57.053673 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37773
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:57.141680 29038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:57.141911 29038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:57.141937 29038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:57.143157 29038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:57.143204 29038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:57.143232 29038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:57.144735 29038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:57.144953 29038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:57.145144 29038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:57.150264 29044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:57.150307 29038 server_base.cc:1034] running on GCE node
W20260806 22:37:57.150277 29045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:57.150283 29049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:57.150794 29038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:57.151080 29038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:57.152263 29038 hybrid_clock.cc:648] HybridClock initialized: now 1786055877152254 us; error 40 us; skew 500 ppm
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:57.155242 29038 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:37:57.155707 29038 webserver.cc:460] Webserver started at http://127.16.235.66:38679/ using document root <none> and password file <none>
I20260806 22:37:57.155942 29038 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:57.155988 29038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:57.156095 29038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:57.157286 29038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "ebd9e3a6e9b34ba19fb0849bc6c44336"
format_stamp: "Formatted at 2026-08-06 22:37:57 on dist-test-slave-3kk6"
server_key: "cdd3b09551f4fcc203393b505e6b8e50"
server_key_iv: "82d952ca5a36f9248d926bc2b26d7c78"
server_key_version: "encryptionkey@0"
I20260806 22:37:57.157630 29038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "ebd9e3a6e9b34ba19fb0849bc6c44336"
format_stamp: "Formatted at 2026-08-06 22:37:57 on dist-test-slave-3kk6"
server_key: "cdd3b09551f4fcc203393b505e6b8e50"
server_key_iv: "82d952ca5a36f9248d926bc2b26d7c78"
server_key_version: "encryptionkey@0"
I20260806 22:37:57.159173 29038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:37:57.160125 29059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.160352 29038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:57.160457 29038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "ebd9e3a6e9b34ba19fb0849bc6c44336"
format_stamp: "Formatted at 2026-08-06 22:37:57 on dist-test-slave-3kk6"
server_key: "cdd3b09551f4fcc203393b505e6b8e50"
server_key_iv: "82d952ca5a36f9248d926bc2b26d7c78"
server_key_version: "encryptionkey@0"
I20260806 22:37:57.160522 29038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:57.175361 29038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:57.177923 29038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:57.178057 29038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:57.178268 29038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:57.178634 29038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:57.178750 29038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.178777 29038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:57.178790 29038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.185024 29038 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:39141
I20260806 22:37:57.185104 29216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:39141 every 8 connection(s)
I20260806 22:37:57.185529 29038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:37:57.190079 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 29038
I20260806 22:37:57.190203 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:37:57.190418 17325 external_mini_cluster.cc:1428] Setting key e7f99abf7bded6e82913117a7441a47a
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:57.196071 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.186421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:48449 (local address 127.16.235.126:37773)
0806 22:37:57.186479 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:57.186481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:57.186805 (+ 324us) server_negotiation.cc:407] Connection header received
0806 22:37:57.187272 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:57.187273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:57.187285 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:57.187352 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:57.188448 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.188977 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.189983 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.190302 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.190378 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.192822 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:57.192823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:57.192824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:57.192832 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:57.194407 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:57.194995 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:57.194996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:57.194997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:57.195056 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:57.195365 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:57.195366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:57.195366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:57.195478 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:57.195529 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:57.195924 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 22:37:57.196001 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:37:57.196472 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.186370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:48449)
0806 22:37:57.186552 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.186599 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:57.187149 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:57.187483 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:57.187487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:57.188132 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:57.188341 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.188347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.189070 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.189071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.189888 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.189891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.190411 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.190412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.190553 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.191180 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:57.191191 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:57.192723 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:57.194503 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:57.194511 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:57.194514 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:57.194908 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:57.195126 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:57.195127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:57.195128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:57.195289 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:57.195544 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:57.195552 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:57.195743 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 22:37:57.195892 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 22:37:57.196108 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260806 22:37:57.197103 29217 heartbeater.cc:347] Connected to a master server at 127.16.235.126:37773
I20260806 22:37:57.197253 29217 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:57.197459 29217 heartbeater.cc:511] Master 127.16.235.126:37773 requested a full tablet report, sending...
I20260806 22:37:57.197937 28767 ts_manager.cc:194] Registered new tserver with Master: ebd9e3a6e9b34ba19fb0849bc6c44336 (127.16.235.66:39141)
I20260806 22:37:57.198518 28767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:48449
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:37:57.220927 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37773
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:57.306614 29224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:57.306931 29224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:57.306975 29224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:57.308298 29224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:57.308396 29224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:57.308449 29224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:57.310096 29224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:57.310449 29224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:57.310772 29224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:57.316154 29233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:57.316187 29235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:57.316187 29238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:57.316567 29224 server_base.cc:1034] running on GCE node
I20260806 22:37:57.316804 29224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:57.317075 29224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:57.318222 29224 hybrid_clock.cc:648] HybridClock initialized: now 1786055877318205 us; error 45 us; skew 500 ppm
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:57.321105 29224 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:37:57.321626 29224 webserver.cc:460] Webserver started at http://127.16.235.67:35689/ using document root <none> and password file <none>
I20260806 22:37:57.321905 29224 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:57.321980 29224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:57.322142 29224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:37:57.323478 29224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "175ffa3a0abb4d7faf7549e0fca907ff"
format_stamp: "Formatted at 2026-08-06 22:37:57 on dist-test-slave-3kk6"
server_key: "b4995382398a02d4cc1a46d841b23e81"
server_key_iv: "b526bbd04ffb5f504f0bcecb1675a410"
server_key_version: "encryptionkey@0"
I20260806 22:37:57.323977 29224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "175ffa3a0abb4d7faf7549e0fca907ff"
format_stamp: "Formatted at 2026-08-06 22:37:57 on dist-test-slave-3kk6"
server_key: "b4995382398a02d4cc1a46d841b23e81"
server_key_iv: "b526bbd04ffb5f504f0bcecb1675a410"
server_key_version: "encryptionkey@0"
I20260806 22:37:57.325565 29224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:57.326417 29252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.326602 29224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:57.326727 29224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "175ffa3a0abb4d7faf7549e0fca907ff"
format_stamp: "Formatted at 2026-08-06 22:37:57 on dist-test-slave-3kk6"
server_key: "b4995382398a02d4cc1a46d841b23e81"
server_key_iv: "b526bbd04ffb5f504f0bcecb1675a410"
server_key_version: "encryptionkey@0"
I20260806 22:37:57.326788 29224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:57.340059 29224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:57.342819 29224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:57.342960 29224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:57.343187 29224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:57.343596 29224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:57.343638 29224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.343662 29224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:57.343719 29224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.350149 29224 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:42755
I20260806 22:37:57.350234 29412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:42755 every 8 connection(s)
I20260806 22:37:57.350601 29224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:57.356909 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 29224
I20260806 22:37:57.356997 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:37:57.357195 17325 external_mini_cluster.cc:1428] Setting key 9eb379a813a028fee6306cf26b9814ab
I20260806 22:37:57.360402 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.351478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:56867 (local address 127.16.235.126:37773)
0806 22:37:57.351546 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:57.351548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:57.351749 (+ 201us) server_negotiation.cc:407] Connection header received
0806 22:37:57.352293 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:57.352294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:57.352305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:57.352360 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:57.353353 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.353883 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.354976 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.355170 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.355227 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.357611 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:57.357613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:57.357613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:57.357622 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:57.359164 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:57.359593 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:57.359594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:57.359594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:57.359653 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:57.359884 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:57.359884 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:57.359885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:57.359998 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:57.360046 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:57.360267 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 22:37:57.360335 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:37:57.360611 29418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.351468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:56867)
0806 22:37:57.351622 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.351671 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:57.352187 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:57.352419 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:57.352423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:57.353052 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:57.353265 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.353270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.353974 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.353974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.354894 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.354898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.355252 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.355253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.355363 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.355882 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:57.355891 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:57.357497 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:37:57.359249 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:57.359256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:57.359257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:57.359519 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:57.359724 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:37:57.359724 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:37:57.359725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:37:57.359827 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:37:57.360042 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:37:57.360045 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:37:57.360165 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 22:37:57.360244 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:37:57.360374 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260806 22:37:57.361167 29413 heartbeater.cc:347] Connected to a master server at 127.16.235.126:37773
I20260806 22:37:57.361321 29413 heartbeater.cc:464] Registering TS with master...
I20260806 22:37:57.361544 29413 heartbeater.cc:511] Master 127.16.235.126:37773 requested a full tablet report, sending...
I20260806 22:37:57.362010 28767 ts_manager.cc:194] Registered new tserver with Master: 175ffa3a0abb4d7faf7549e0fca907ff (127.16.235.67:42755)
I20260806 22:37:57.362406 28767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:56867
I20260806 22:37:57.369299 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab.
Entry for principal oryx/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab.
WARNING: no policy specified for oryx/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab.
Entry for principal oryx/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab.
WARNING: no policy specified for oryx/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab.
Entry for principal oryx/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab.
I20260806 22:37:57.454366 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 28851
I20260806 22:37:57.464980 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 29038
I20260806 22:37:57.478915 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 29224
I20260806 22:37:57.493841 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 28724
I20260806 22:37:57.507074 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37773
--webserver_interface=127.16.235.126
--webserver_port=36739
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:37773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:37:57.595796 29426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:57.596043 29426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:57.596068 29426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:57.597306 29426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:37:57.597359 29426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:57.597374 29426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:57.597388 29426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:37:57.597445 29426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:37:57.599099 29426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:37773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:37773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=36739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:57.599406 29426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:57.599758 29426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:57.605129 29438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:57.605186 29436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:57.605147 29434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:57.605343 29426 server_base.cc:1034] running on GCE node
I20260806 22:37:57.605628 29426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:57.605963 29426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:57.607136 29426 hybrid_clock.cc:648] HybridClock initialized: now 1786055877607109 us; error 62 us; skew 500 ppm
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055877, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:57.610067 29426 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:37:57.610560 29426 webserver.cc:460] Webserver started at http://127.16.235.126:36739/ using document root <none> and password file <none>
I20260806 22:37:57.610863 29426 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:57.610967 29426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:57.612457 29426 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:57.613281 29448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.613497 29426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:57.613616 29426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "0dd49bb3aeee4d36ba12eb48cc190009"
format_stamp: "Formatted at 2026-08-06 22:37:56 on dist-test-slave-3kk6"
server_key: "87aed152c4f589c37fa120ac62f36fb9"
server_key_iv: "e6bc812727c8007787909f51c087d18d"
server_key_version: "encryptionkey@0"
I20260806 22:37:57.614018 29426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:57.627475 29426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:57.630232 29426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:57.630438 29426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:57.635083 29426 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:37773
I20260806 22:37:57.635169 29523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:37773 every 8 connection(s)
I20260806 22:37:57.635612 29426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:37:57.637347 29525 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:37:57.638108 29525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: Bootstrap starting.
I20260806 22:37:57.643393 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 29426
I20260806 22:37:57.643759 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:38177
--local_ip_for_outbound_sockets=127.16.235.65
--tserver_master_addrs=127.16.235.126:37773
--webserver_port=43025
--webserver_interface=127.16.235.65
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.235.65
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:37:57.647616 29525 log.cc:824] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: Log is configured to *not* fsync() on all Append() calls
I20260806 22:37:57.649518 29525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:37:57.649857 29525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: Bootstrap complete.
I20260806 22:37:57.652386 29525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } }
I20260806 22:37:57.652683 29525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dd49bb3aeee4d36ba12eb48cc190009, State: Initialized, Role: FOLLOWER
I20260806 22:37:57.652907 29525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } }
I20260806 22:37:57.653016 29525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:37:57.653077 29525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:37:57.653129 29525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:37:57.654277 29525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } }
I20260806 22:37:57.654417 29525 leader_election.cc:302] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dd49bb3aeee4d36ba12eb48cc190009; no voters:
I20260806 22:37:57.654696 29525 leader_election.cc:288] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:37:57.654768 29530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:37:57.655009 29530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [term 2 LEADER]: Becoming Leader. State: Replica: 0dd49bb3aeee4d36ba12eb48cc190009, State: Running, Role: LEADER
I20260806 22:37:57.654991 29525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:37:57.655779 29530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } }
I20260806 22:37:57.656697 29533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dd49bb3aeee4d36ba12eb48cc190009. Latest consensus state: current_term: 2 leader_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } } }
I20260806 22:37:57.656782 29533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:57.656502 29532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd49bb3aeee4d36ba12eb48cc190009" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 37773 } } }
I20260806 22:37:57.657027 29532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009 [sys.catalog]: This master's current role is: LEADER
I20260806 22:37:57.657341 29553 catalog_manager.cc:1259] Loaded cluster ID: 2fb0a22cd33c4b4f8806daf4b8aee40f
I20260806 22:37:57.657407 29553 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: loading cluster ID for follower catalog manager: success
I20260806 22:37:57.658381 29553 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: acquiring CA information for follower catalog manager: success
I20260806 22:37:57.658797 29553 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:37:57.658876 29556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:37:57.659024 29556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:37:57.659072 29556 catalog_manager.cc:1259] Loaded cluster ID: 2fb0a22cd33c4b4f8806daf4b8aee40f
I20260806 22:37:57.659087 29556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:37:57.659215 29556 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:37:57.659261 29556 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0dd49bb3aeee4d36ba12eb48cc190009: Loaded TSK: 0
I20260806 22:37:57.659338 29556 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:37:57.751822 29528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:57.752065 29528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:57.752094 29528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:57.753330 29528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:57.753378 29528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:57.753407 29528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:37:57.755048 29528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.65.keytab
--principal=oryx/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:38177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=43025
--tserver_master_addrs=127.16.235.126:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:57.755311 29528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:57.755577 29528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:57.762054 29573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:57.762095 29568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:57.762270 29528 server_base.cc:1034] running on GCE node
W20260806 22:37:57.762373 29569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:57.762617 29528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:57.762935 29528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:57.764163 29528 hybrid_clock.cc:648] HybridClock initialized: now 1786055877764145 us; error 34 us; skew 500 ppm
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055877, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:57.767282 29528 init.cc:377] Logged in from keytab as oryx/127.16.235.65@KRBTEST.COM (short username oryx)
I20260806 22:37:57.767872 29528 webserver.cc:460] Webserver started at http://127.16.235.65:43025/ using document root <none> and password file <none>
I20260806 22:37:57.768142 29528 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:57.768209 29528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:57.769802 29528 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:57.770522 29587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.770718 29528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:37:57.770833 29528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "5ea7c46d18f64c2ab312fd567cd0f4ae"
format_stamp: "Formatted at 2026-08-06 22:37:56 on dist-test-slave-3kk6"
server_key: "88edf794e06f850421a2a65feb77f0b9"
server_key_iv: "ab8778a47a78af2758357d6b346750fc"
server_key_version: "encryptionkey@0"
I20260806 22:37:57.771227 29528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:57.781172 29528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:57.783802 29528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:57.783975 29528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:57.784319 29528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:57.784751 29528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:57.784790 29528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.784814 29528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:57.784826 29528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.792933 29528 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:38177
I20260806 22:37:57.793030 29746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:38177 every 8 connection(s)
I20260806 22:37:57.793427 29528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:37:57.800781 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 29528
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:57.801147 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:39141
--local_ip_for_outbound_sockets=127.16.235.66
--tserver_master_addrs=127.16.235.126:37773
--webserver_port=38679
--webserver_interface=127.16.235.66
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.235.66
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:57.803074 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.794372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:37955)
0806 22:37:57.794574 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.794627 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:57.795197 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:57.795682 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:57.795687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:57.796321 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:57.796524 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.796530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.797724 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.797726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.798543 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.798546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.799213 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.799214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.799358 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.799968 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:57.799978 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:57.802814 (+ 2836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":96,"threads_started":1}
W20260806 22:37:57.803325 29751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:57.794430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:37955 (local address 127.16.235.126:37773)
0806 22:37:57.794694 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:57.794702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:57.794764 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:37:57.795445 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:57.795450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:57.795495 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:57.795626 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:57.796658 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.797602 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.798686 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.799113 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.799240 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.803128 (+ 3888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:37955: BlockingRecv error: recv got EOF from 127.16.235.65:37955 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":114,"threads_started":1}
W20260806 22:37:57.803645 29748 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:37773 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.235.126:37773: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:37:57.809998 29751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:57.803979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:55221 (local address 127.16.235.126:37773)
0806 22:37:57.804017 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:57.804019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:57.804028 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:57.804113 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:57.804114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:57.804124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:57.804189 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:57.804482 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.805050 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.806097 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.806405 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.806452 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.809900 (+ 3448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:55221: BlockingRecv error: recv got EOF from 127.16.235.65:55221 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:37:57.810410 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.803938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:55221)
0806 22:37:57.803987 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.803998 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:57.804042 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:57.804193 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:57.804194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:57.804339 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:57.804396 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.804398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.805149 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.805151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.805989 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.805993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.806545 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.806546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.806629 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.807212 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:57.807218 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:57.810312 (+ 3094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:57.816890 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.810981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:42363)
0806 22:37:57.811151 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.811164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:57.811248 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:57.811451 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:57.811452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:57.811628 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:57.811706 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.811709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.812770 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.812771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.813673 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.813678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.814285 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.814286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.814348 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.814924 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:57.814931 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:57.816835 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151}
W20260806 22:37:57.817020 29751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:57.811071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:42363 (local address 127.16.235.126:37773)
0806 22:37:57.811116 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:57.811119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:57.811215 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:37:57.811329 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:57.811330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:57.811344 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:57.811379 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:57.811816 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.812653 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.813791 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.814136 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.814190 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.816950 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:42363: BlockingRecv error: recv got EOF from 127.16.235.65:42363 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:37:57.817093 29748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:37:57.893285 29752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:57.893534 29752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:57.893560 29752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:57.894795 29752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:57.894840 29752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:57.894867 29752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:37:57.896306 29752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.66.keytab
--principal=oryx/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:39141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=38679
--tserver_master_addrs=127.16.235.126:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:57.896521 29752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:57.896715 29752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:57.902153 29763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:57.902191 29760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:57.902345 29752 server_base.cc:1034] running on GCE node
W20260806 22:37:57.902191 29759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:57.902707 29752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:57.902952 29752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:57.904109 29752 hybrid_clock.cc:648] HybridClock initialized: now 1786055877904083 us; error 45 us; skew 500 ppm
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055877, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:57.906848 29752 init.cc:377] Logged in from keytab as oryx/127.16.235.66@KRBTEST.COM (short username oryx)
I20260806 22:37:57.907331 29752 webserver.cc:460] Webserver started at http://127.16.235.66:38679/ using document root <none> and password file <none>
I20260806 22:37:57.907581 29752 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:57.907668 29752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:57.909183 29752 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:37:57.909830 29774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.909987 29752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:37:57.910097 29752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "ebd9e3a6e9b34ba19fb0849bc6c44336"
format_stamp: "Formatted at 2026-08-06 22:37:57 on dist-test-slave-3kk6"
server_key: "cdd3b09551f4fcc203393b505e6b8e50"
server_key_iv: "82d952ca5a36f9248d926bc2b26d7c78"
server_key_version: "encryptionkey@0"
I20260806 22:37:57.910481 29752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:57.920493 29752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:57.922535 29752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:57.922745 29752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:57.923031 29752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:57.923369 29752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:57.923425 29752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.923468 29752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:57.923511 29752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:57.929808 29752 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:39141
I20260806 22:37:57.929860 29935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:39141 every 8 connection(s)
I20260806 22:37:57.930258 29752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:57.937908 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.931118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:50839)
0806 22:37:57.931307 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.931353 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:57.931901 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:57.932136 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:57.932139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:57.932779 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:57.932989 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.932994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.933661 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.933661 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.934468 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.934470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.934896 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.934897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.935009 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.935518 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:57.935527 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:57.937744 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
W20260806 22:37:57.937973 29751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:57.931138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:50839 (local address 127.16.235.126:37773)
0806 22:37:57.931214 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:57.931216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:57.931448 (+ 232us) server_negotiation.cc:407] Connection header received
0806 22:37:57.932000 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:57.932001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:57.932017 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:57.932080 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:57.933080 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.933575 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.934550 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.934800 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.934858 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.937922 (+ 3064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:50839: BlockingRecv error: recv got EOF from 127.16.235.66:50839 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 22:37:57.938297 29936 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:37773 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.235.126:37773: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
I20260806 22:37:57.939317 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 29752
I20260806 22:37:57.939643 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:42755
--local_ip_for_outbound_sockets=127.16.235.67
--tserver_master_addrs=127.16.235.126:37773
--webserver_port=35689
--webserver_interface=127.16.235.67
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.235.67
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:57.943410 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.938618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:48411)
0806 22:37:57.938697 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.938712 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:57.938765 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:57.938899 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:57.938900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:57.939023 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:57.939076 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.939077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.939809 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.939811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.940332 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.940334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.940812 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.940813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.940867 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.941383 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:57.941388 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:57.943356 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 22:37:57.943565 29751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:57.938716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:48411 (local address 127.16.235.126:37773)
0806 22:37:57.938780 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:57.938782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:57.938794 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:37:57.938824 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:57.938824 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:57.938833 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:57.938879 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:57.939139 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.939669 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.940422 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.940726 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.940809 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.943483 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:48411: BlockingRecv error: recv got EOF from 127.16.235.66:48411 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:57 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:57.949322 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:57.943880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:60375)
0806 22:37:57.944002 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:57.944014 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:57.944070 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:57.944194 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:57.944195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:57.944378 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:57.944422 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.944424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.945123 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.945124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.946037 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:57.946040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.946412 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.946413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:57.946479 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.947037 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:57.947043 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:57.949219 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102}
W20260806 22:37:57.949388 29751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:57.943951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:60375 (local address 127.16.235.126:37773)
0806 22:37:57.944006 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:57.944008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:57.944063 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:37:57.944129 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:57.944129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:57.944138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:57.944185 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:57.944498 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.945035 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.946137 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:57.946328 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:57.946388 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:57.949314 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:60375: BlockingRecv error: recv got EOF from 127.16.235.66:60375 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 22:37:57.949604 29936 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:37:58.028228 29941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:37:58.028467 29941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:37:58.028494 29941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:37:58.029707 29941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:37:58.029757 29941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:37:58.029784 29941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:37:58.031422 29941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/krb5kdc/oryx_127.16.235.67.keytab
--principal=oryx/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:42755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=35689
--tserver_master_addrs=127.16.235.126:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:37:58.031708 29941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:37:58.031973 29941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:37:58.037309 29956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:58.037309 29950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:37:58.037302 29952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:37:58.037572 29941 server_base.cc:1034] running on GCE node
I20260806 22:37:58.037803 29941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:37:58.038012 29941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:37:58.039218 29941 hybrid_clock.cc:648] HybridClock initialized: now 1786055878039191 us; error 39 us; skew 500 ppm
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055878, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:37:58.042122 29941 init.cc:377] Logged in from keytab as oryx/127.16.235.67@KRBTEST.COM (short username oryx)
I20260806 22:37:58.042774 29941 webserver.cc:460] Webserver started at http://127.16.235.67:35689/ using document root <none> and password file <none>
I20260806 22:37:58.043041 29941 fs_manager.cc:359] Metadata directory not provided
I20260806 22:37:58.043128 29941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:37:58.044664 29941 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:37:58.045428 29963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:58.045652 29941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:37:58.045737 29941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "175ffa3a0abb4d7faf7549e0fca907ff"
format_stamp: "Formatted at 2026-08-06 22:37:57 on dist-test-slave-3kk6"
server_key: "b4995382398a02d4cc1a46d841b23e81"
server_key_iv: "b526bbd04ffb5f504f0bcecb1675a410"
server_key_version: "encryptionkey@0"
I20260806 22:37:58.046125 29941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:37:58.067040 29941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:37:58.069339 29941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:37:58.069473 29941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:37:58.069705 29941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:37:58.070083 29941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:37:58.070119 29941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:58.070142 29941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:37:58.070194 29941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:37:58.076404 29941 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:42755
I20260806 22:37:58.076488 30125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:42755 every 8 connection(s)
I20260806 22:37:58.076869 29941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:37:58.077234 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 29941
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:37:58.085680 29751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:58.077741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:43305 (local address 127.16.235.126:37773)
0806 22:37:58.077899 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:58.077901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:58.078137 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:37:58.078630 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:58.078632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:58.078641 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:58.078768 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:58.080099 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.080750 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.082053 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.082218 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.082296 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.085624 (+ 3328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:43305: BlockingRecv error: recv got EOF from 127.16.235.67:43305 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 22:37:58.085721 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:58.077732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:43305)
0806 22:37:58.077980 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:58.078029 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:58.078539 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:58.078886 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:58.078891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:58.079772 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:58.080003 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.080007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.080836 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.080837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.081959 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.081961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.082309 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.082311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.082438 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.082906 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:58.082915 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:58.085465 (+ 2550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":98,"threads_started":1}
W20260806 22:37:58.085934 30128 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:37773 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.235.126:37773: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
I20260806 22:37:58.087083 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:58.077903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.16.235.126:37773)
0806 22:37:58.078022 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:58.078023 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:58.078861 (+ 838us) server_negotiation.cc:407] Connection header received
0806 22:37:58.078921 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:58.078921 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:58.078929 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:58.078982 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:58.079825 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.080570 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.081662 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.081861 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.081940 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.083449 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:37:58.083454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:37:58.083454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:37:58.083468 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:37:58.085348 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:58.085736 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:58.085740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:58.085740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:58.085943 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:37:58.086176 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:37:58.086177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:37:58.086178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:37:58.086548 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:37:58.086725 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 22:37:58.086862 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 22:37:58.086983 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:58.090826 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:58.086208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:36525)
0806 22:37:58.086327 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:58.086337 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:58.086371 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:58.086508 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:58.086508 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:58.086663 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:58.086742 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.086744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.087530 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.087531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.088302 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.088305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.088621 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.088621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.088667 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.089039 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:58.089044 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:58.090772 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260806 22:37:58.090971 29751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:58.086277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:36525 (local address 127.16.235.126:37773)
0806 22:37:58.086350 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:58.086352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:58.086361 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:58.086427 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:58.086428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:58.086439 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:58.086504 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:58.086805 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.087459 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.088372 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.088549 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.088605 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.090896 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:36525: BlockingRecv error: recv got EOF from 127.16.235.67:36525 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:58.095453 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:58.091124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:50059)
0806 22:37:58.091156 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:58.091167 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:58.091210 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:58.091340 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:58.091341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:58.091474 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:58.091500 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.091502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.092138 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.092139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.092635 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.092636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.092998 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.092999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.093049 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.093445 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:58.093450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:58.095353 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 22:37:58.095556 29751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:58.091156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:50059 (local address 127.16.235.126:37773)
0806 22:37:58.091204 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:58.091206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:58.091215 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:37:58.091272 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:58.091272 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:58.091281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:58.091337 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:58.091580 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.092051 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.092737 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.092925 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.093000 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.095462 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:50059: BlockingRecv error: recv got EOF from 127.16.235.67:50059 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:37:58.095741 30128 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:58.823582 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:58.817479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:58805)
0806 22:37:58.817654 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:58.817666 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:58.817770 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:58.817918 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:58.817919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:58.818165 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:58.818224 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.818227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.819034 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.819036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.819596 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.819598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.820054 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.820055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.820099 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.820820 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:58.820826 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:58.823460 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":107,"threads_started":1}
W20260806 22:37:58.823668 30137 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:58.817558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:58805 (local address 127.16.235.126:37773)
0806 22:37:58.817690 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:58.817692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:58.817768 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:37:58.817823 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:58.817824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:58.817835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:58.817902 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:58.818320 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.818935 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.819734 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.819970 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.820044 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.823574 (+ 3530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:58805: BlockingRecv error: recv got EOF from 127.16.235.65:58805 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:58 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:58.955447 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:58.950136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:47413)
0806 22:37:58.950326 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:58.950338 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:58.950428 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:58.950592 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:58.950593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:58.950864 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:58.950918 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.950920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.951625 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.951626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.952220 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:58.952222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.952653 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.952654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:58.952731 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.953251 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:58.953257 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:58.955370 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":81,"threads_started":1}
W20260806 22:37:58.955610 30137 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:58.950249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:47413 (local address 127.16.235.126:37773)
0806 22:37:58.950327 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:58.950329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:58.950415 (+ 86us) server_negotiation.cc:407] Connection header received
0806 22:37:58.950490 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:58.950491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:58.950506 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:58.950587 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:58.951015 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.951531 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.952357 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:58.952559 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:58.952653 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:58.955532 (+ 2879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:47413: BlockingRecv error: recv got EOF from 127.16.235.66:47413 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:37:59 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:59 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:59.101904 30140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:59.096180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:46457)
0806 22:37:59.096368 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:59.096381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:59.096466 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:59.096612 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:59.096613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:59.096843 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:59.096898 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:59.096900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.097589 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.097590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:59.098270 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:59.098272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.098781 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.098782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:59.098864 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:59.099397 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:59.099402 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:59.101811 (+ 2409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
W20260806 22:37:59.102075 30137 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:59.096304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:46457 (local address 127.16.235.126:37773)
0806 22:37:59.096353 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:59.096356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:59.096463 (+ 107us) server_negotiation.cc:407] Connection header received
0806 22:37:59.096522 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:59.096522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:59.096535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:59.096586 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:59.096975 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.097494 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.098370 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.098563 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.098626 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:59.101986 (+ 3360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:46457: BlockingRecv error: recv got EOF from 127.16.235.67:46457 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 22:37:59 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:59 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:59.829747 30141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:59.824244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:32795)
0806 22:37:59.824421 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:59.824435 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:59.824507 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:59.824785 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:59.824786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:59.824995 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:59.825067 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:59.825069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.825966 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.825968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:59.826535 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:59.826537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.827016 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.827017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:59.827072 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:59.827644 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:59.827648 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:59.829685 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":106,"threads_started":1}
W20260806 22:37:59.829967 30142 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:59.824379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:32795 (local address 127.16.235.126:37773)
0806 22:37:59.824533 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:59.824535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:59.824546 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:37:59.824596 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:59.824597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:59.824614 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:59.824746 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:59.825168 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.825864 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.826637 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.826898 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.826963 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:59.829867 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:32795: BlockingRecv error: recv got EOF from 127.16.235.65:32795 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
Aug 06 22:37:59 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:37:59 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:37:59.961472 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:37:59.956148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:37025)
0806 22:37:59.956306 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:37:59.956325 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:37:59.956399 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:37:59.956580 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:37:59.956581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:37:59.956770 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:37:59.956856 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:59.956862 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.957535 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.957536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:59.958074 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:37:59.958075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.958593 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.958594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:37:59.958697 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:59.959433 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:37:59.959438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:37:59.961407 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
W20260806 22:37:59.961648 30142 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:37:59.956234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:37025 (local address 127.16.235.126:37773)
0806 22:37:59.956311 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:37:59.956313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:37:59.956381 (+ 68us) server_negotiation.cc:407] Connection header received
0806 22:37:59.956455 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:37:59.956456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:37:59.956468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:37:59.956538 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:37:59.956937 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.957453 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.958197 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:37:59.958464 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:37:59.958548 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:37:59.961579 (+ 3031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:37025: BlockingRecv error: recv got EOF from 127.16.235.66:37025 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:38:00 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:00 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:00.108865 30145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:00.102534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:39003)
0806 22:38:00.102733 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:00.102765 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:00.102856 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:00.103624 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:00.103625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:00.103876 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:00.103945 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:00.103948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.104856 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.104857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:00.105536 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:00.105539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.105940 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.105941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:00.105994 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:00.106610 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:00.106616 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:00.108764 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":87,"threads_started":1}
W20260806 22:38:00.108948 30142 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:00.103328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:39003 (local address 127.16.235.126:37773)
0806 22:38:00.103380 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:00.103382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:00.103425 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:38:00.103460 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:00.103461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:00.103478 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:00.103551 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:00.104027 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.104731 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.105615 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.105807 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.105893 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:00.108880 (+ 2987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:39003: BlockingRecv error: recv got EOF from 127.16.235.67:39003 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 22:38:00 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:00 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:00.835667 30147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:00.830377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:49105)
0806 22:38:00.830562 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:00.830574 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:00.830637 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:00.830850 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:00.830851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:00.831040 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:00.831095 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:00.831096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.831844 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.831844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:00.832382 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:00.832384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.832812 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.832813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:00.832856 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:00.833434 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:00.833440 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:00.835562 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":111,"threads_started":1}
W20260806 22:38:00.835748 30150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:00.830485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:49105 (local address 127.16.235.126:37773)
0806 22:38:00.830606 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:00.830610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:00.830626 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:38:00.830754 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:00.830754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:00.830766 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:00.830825 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:00.831169 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.831750 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.832475 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.832694 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.832783 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:00.835669 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:49105: BlockingRecv error: recv got EOF from 127.16.235.65:49105 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
Aug 06 22:38:00 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:00 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:00.967298 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:00.962105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:52011)
0806 22:38:00.962316 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:00.962331 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:00.962429 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:00.962603 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:00.962604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:00.962865 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:00.962915 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:00.962917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.963633 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.963634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:00.964189 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:00.964191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.964594 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.964594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:00.964653 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:00.965230 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:00.965235 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:00.967184 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":99,"threads_started":1}
W20260806 22:38:00.967360 30150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:00.962217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:52011 (local address 127.16.235.126:37773)
0806 22:38:00.962302 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:00.962306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:00.962407 (+ 101us) server_negotiation.cc:407] Connection header received
0806 22:38:00.962503 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:00.962504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:00.962518 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:00.962578 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:00.963011 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.963536 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.964289 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:00.964510 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:00.964595 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:00.967292 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:52011: BlockingRecv error: recv got EOF from 127.16.235.66:52011 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 22:38:01 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:01 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:01.115250 30152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:01.109580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:50123)
0806 22:38:01.109781 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:01.109802 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:01.109873 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:01.110074 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:01.110075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:01.110274 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:01.110335 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:01.110336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.111157 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.111157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:01.111769 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:01.111771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.112151 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.112152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:01.112193 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:01.112855 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:01.112863 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:01.115147 (+ 2284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":141,"threads_started":1}
W20260806 22:38:01.115316 30150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:01.109720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:50123 (local address 127.16.235.126:37773)
0806 22:38:01.109801 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:01.109803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:01.109873 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:38:01.109961 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:01.109962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:01.109976 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:01.110027 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:01.110429 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.111053 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.111873 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.112076 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.112143 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:01.115251 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:50123: BlockingRecv error: recv got EOF from 127.16.235.67:50123 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 22:38:01 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:01 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:01.841663 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:01.836307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:40019)
0806 22:38:01.836505 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:01.836516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:01.836585 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:01.836741 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:01.836742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:01.836936 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:01.836989 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:01.836991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.837702 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.837703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:01.838254 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:01.838256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.838611 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.838612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:01.838718 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:01.839410 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:01.839417 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:01.841561 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":135,"threads_started":1}
W20260806 22:38:01.841730 30155 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:01.836408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:40019 (local address 127.16.235.126:37773)
0806 22:38:01.836535 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:01.836537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:01.836568 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:38:01.836648 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:01.836648 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:01.836659 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:01.836721 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:01.837078 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.837617 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.838340 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.838538 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.838594 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:01.841656 (+ 3062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:40019: BlockingRecv error: recv got EOF from 127.16.235.65:40019 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
Aug 06 22:38:01 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:01 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:01.972826 30157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:01.967895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:35591)
0806 22:38:01.968054 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:01.968071 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:01.968138 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:01.968290 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:01.968291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:01.968488 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:01.968542 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:01.968543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.969270 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.969272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:01.969809 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:01.969812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.970173 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.970175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:01.970231 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:01.970852 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:01.970857 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:01.972762 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
W20260806 22:38:01.972920 30155 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:01.967988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:35591 (local address 127.16.235.126:37773)
0806 22:38:01.968074 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:01.968076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:01.968137 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:38:01.968201 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:01.968201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:01.968214 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:01.968274 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:01.968638 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.969152 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.969891 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:01.970079 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:01.970137 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:01.972871 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:35591: BlockingRecv error: recv got EOF from 127.16.235.66:35591 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 22:38:02 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:02 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:02.125234 30158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:02.115840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:52939)
0806 22:38:02.116075 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:02.116087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:02.116177 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:02.116362 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:02.116363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:02.116555 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:02.116601 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:02.116603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.117354 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.117355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:02.117901 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:02.117903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.118310 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.118311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:02.118368 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:02.122974 (+ 4606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:02.122979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:02.125170 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":129,"threads_started":1}
W20260806 22:38:02.125476 30155 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:02.115974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:52939 (local address 127.16.235.126:37773)
0806 22:38:02.116049 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:02.116051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:02.116156 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:38:02.116260 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:02.116261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:02.116281 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:02.116354 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:02.116696 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.117254 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.117994 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.118234 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.118318 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:02.125415 (+ 7097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:52939: BlockingRecv error: recv got EOF from 127.16.235.67:52939 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 22:38:02 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:02 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:02.847260 30159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:02.842264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:33245)
0806 22:38:02.842428 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:02.842442 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:02.842524 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:02.842718 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:02.842719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:02.842926 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:02.842981 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:02.842983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.843772 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.843772 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:02.844304 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:02.844305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.844645 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.844646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:02.844688 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:02.845161 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:02.845166 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:02.847197 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
W20260806 22:38:02.847466 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:02.842304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:33245 (local address 127.16.235.126:37773)
0806 22:38:02.842442 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:02.842445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:02.842503 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:38:02.842580 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:02.842581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:02.842594 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:02.842745 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:02.843101 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.843681 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.844382 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.844575 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.844629 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:02.847379 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:33245: BlockingRecv error: recv got EOF from 127.16.235.65:33245 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
Aug 06 22:38:02 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:02 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:02.978613 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:02.973484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:48841)
0806 22:38:02.973694 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:02.973711 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:02.973795 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:02.973962 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:02.973963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:02.974146 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:02.974200 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:02.974202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.975214 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.975215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:02.975757 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:02.975759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.976080 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.976081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:02.976121 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:02.976606 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:02.976610 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:02.978511 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
W20260806 22:38:02.978688 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:02.973664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:48841 (local address 127.16.235.126:37773)
0806 22:38:02.973731 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:02.973735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:02.973773 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:38:02.973865 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:02.973866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:02.973878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:02.973933 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:02.974316 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.975138 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.975838 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:02.976018 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:02.976067 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:02.978596 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:48841: BlockingRecv error: recv got EOF from 127.16.235.66:48841 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 22:38:03 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:03 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:03.131425 30164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:03.125914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:51695)
0806 22:38:03.126150 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:03.126165 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:03.126243 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:03.126448 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:03.126449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:03.126640 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:03.126757 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:03.126759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.127494 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.127495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:03.128037 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:03.128039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.128397 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.128398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:03.128440 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:03.128951 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:03.128956 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:03.131346 (+ 2390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":112,"threads_started":1}
W20260806 22:38:03.131500 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:03.126063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:51695 (local address 127.16.235.126:37773)
0806 22:38:03.126155 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:03.126157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:03.126238 (+ 81us) server_negotiation.cc:407] Connection header received
0806 22:38:03.126294 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:03.126296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:03.126310 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:03.126374 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:03.126865 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.127392 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.128126 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.128324 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.128383 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:03.131450 (+ 3067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:51695: BlockingRecv error: recv got EOF from 127.16.235.67:51695 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 22:38:03 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:03 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:03.853204 30166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:03.847854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:59667)
0806 22:38:03.848058 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:03.848071 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:03.848151 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:03.848404 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:03.848405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:03.848639 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:03.848722 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:03.848725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.849467 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.849469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:03.850054 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:03.850056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.850450 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.850451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:03.850491 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:03.851081 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:03.851085 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:03.853142 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":133,"threads_started":1}
W20260806 22:38:03.853359 30167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:03.847966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:59667 (local address 127.16.235.126:37773)
0806 22:38:03.848090 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:03.848092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:03.848147 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:38:03.848215 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:03.848216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:03.848228 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:03.848352 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:03.848801 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.849324 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.850169 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.850375 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.850446 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:03.853260 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:59667: BlockingRecv error: recv got EOF from 127.16.235.65:59667 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
Aug 06 22:38:03 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:03 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:03.984233 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:03.979185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:55565)
0806 22:38:03.979341 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:03.979353 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:03.979432 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:03.979566 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:03.979566 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:03.979753 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:03.979812 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:03.979814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.980499 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.980500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:03.981061 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:03.981063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.981419 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.981420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:03.981461 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:03.981969 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:03.981976 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:03.984170 (+ 2194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
W20260806 22:38:03.984371 30167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:03.979253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:55565 (local address 127.16.235.126:37773)
0806 22:38:03.979332 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:03.979341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:03.979443 (+ 102us) server_negotiation.cc:407] Connection header received
0806 22:38:03.979487 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:03.979488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:03.979502 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:03.979548 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:03.979889 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.980400 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.981152 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:03.981347 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:03.981414 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:03.984284 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:55565: BlockingRecv error: recv got EOF from 127.16.235.66:55565 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:38:04 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:04 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:04.136991 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:04.131969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:41547)
0806 22:38:04.132210 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:04.132221 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:04.132307 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:04.132465 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:04.132465 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:04.132672 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:04.132721 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:04.132723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.133424 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.133425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:04.133960 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:04.133961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.134372 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.134373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:04.134439 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:04.135019 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:04.135024 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:04.136929 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":77,"threads_started":1}
W20260806 22:38:04.137162 30167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:04.132036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:41547 (local address 127.16.235.126:37773)
0806 22:38:04.132116 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:04.132118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:04.132299 (+ 181us) server_negotiation.cc:407] Connection header received
0806 22:38:04.132366 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:04.132367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:04.132382 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:04.132458 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:04.132824 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.133337 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.134054 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.134294 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.134373 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:04.137085 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:41547: BlockingRecv error: recv got EOF from 127.16.235.67:41547 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:38:04 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:04 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:04.858857 30172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:04.853747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:46855)
0806 22:38:04.853904 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:04.853916 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:04.853972 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:04.854145 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:04.854146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:04.854358 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:04.854420 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:04.854422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.855147 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.855148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:04.855748 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:04.855750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.856141 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.856142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:04.856202 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:04.856716 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:04.856720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:04.858737 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
W20260806 22:38:04.858909 30173 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:04.853873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:46855 (local address 127.16.235.126:37773)
0806 22:38:04.854009 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:04.854011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:04.854020 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:38:04.854043 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:04.854044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:04.854054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:04.854130 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:04.854489 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.855034 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.855823 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.856064 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.856118 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:04.858835 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:46855: BlockingRecv error: recv got EOF from 127.16.235.65:46855 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
Aug 06 22:38:04 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:04 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:04.990788 30175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:04.984918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:42459)
0806 22:38:04.985085 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:04.985098 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:04.985186 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:04.985356 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:04.985357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:04.985544 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:04.985596 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:04.985599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.986418 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.986420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:04.987448 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:04.987452 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.987922 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.987923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:04.987977 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:04.988553 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:04.988558 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:04.990622 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
W20260806 22:38:04.990959 30173 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:04.985030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:42459 (local address 127.16.235.126:37773)
0806 22:38:04.985105 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:04.985107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:04.985162 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:38:04.985245 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:04.985246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:04.985257 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:04.985340 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:04.985676 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.986210 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.987569 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:04.987796 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:04.987889 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:04.990848 (+ 2959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:42459: BlockingRecv error: recv got EOF from 127.16.235.66:42459 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:38:05 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:05 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:05.143301 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:05.137620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:59757)
0806 22:38:05.137888 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:05.137900 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:05.137966 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:05.138123 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:05.138124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:05.138306 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:05.138366 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:05.138367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.139121 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.139123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:05.139741 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:05.139744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.140192 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.140193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:05.140245 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:05.140915 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:05.140921 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:05.143221 (+ 2300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":157,"threads_started":1}
W20260806 22:38:05.143504 30173 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:05.137712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:59757 (local address 127.16.235.126:37773)
0806 22:38:05.137825 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:05.137827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:05.137968 (+ 141us) server_negotiation.cc:407] Connection header received
0806 22:38:05.138034 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:05.138035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:05.138049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:05.138098 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:05.138439 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.139007 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.139885 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.140086 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.140156 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:05.143417 (+ 3261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:59757: BlockingRecv error: recv got EOF from 127.16.235.67:59757 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 06 22:38:05 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:05 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:05.864694 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:05.859395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:44841)
0806 22:38:05.859573 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:05.859586 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:05.859649 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:05.859796 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:05.859797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:05.859994 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:05.860048 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:05.860050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.860885 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.860886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:05.861429 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:05.861431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.861861 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.861861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:05.861904 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:05.862493 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:05.862498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:05.864575 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
W20260806 22:38:05.864916 30178 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:05.859507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:44841 (local address 127.16.235.126:37773)
0806 22:38:05.859626 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:05.859628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:05.859636 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:38:05.859705 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:05.859705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:05.859717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:05.859777 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:05.860159 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.860765 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.861543 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.861780 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.861851 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:05.864812 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:44841: BlockingRecv error: recv got EOF from 127.16.235.65:44841 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
Aug 06 22:38:05 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:05 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:05.996984 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:05.991532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:48771)
0806 22:38:05.991741 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:05.991753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:05.991823 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:05.992010 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:05.992012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:05.992319 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:05.992388 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:05.992391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.993432 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.993433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:05.994000 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:05.994003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.994370 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.994371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:05.994427 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:05.994956 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:05.994960 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:05.996919 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
W20260806 22:38:05.997140 30178 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:05.991606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:48771 (local address 127.16.235.126:37773)
0806 22:38:05.991714 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:05.991717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:05.991824 (+ 107us) server_negotiation.cc:407] Connection header received
0806 22:38:05.991886 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:05.991887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:05.991900 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:05.991963 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:05.992496 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.993332 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.994089 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:05.994275 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:05.994330 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:05.997053 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:48771: BlockingRecv error: recv got EOF from 127.16.235.66:48771 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 22:38:06 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:06 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:06.149796 30182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:06.144162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:38785)
0806 22:38:06.144354 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:06.144368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:06.144437 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:06.144733 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:06.144735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:06.144948 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:06.145014 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:06.145016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.145790 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.145791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:06.146394 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:06.146396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.146814 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.146815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:06.146868 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:06.147393 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:06.147398 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:06.149697 (+ 2299us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":122,"threads_started":1}
W20260806 22:38:06.149921 30178 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:06.144312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:38785 (local address 127.16.235.126:37773)
0806 22:38:06.144390 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:06.144391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:06.144423 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:38:06.144573 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:06.144575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:06.144592 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:06.144664 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:06.145139 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.145678 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.146481 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.146691 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.146748 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:06.149827 (+ 3079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:38785: BlockingRecv error: recv got EOF from 127.16.235.67:38785 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:38:06 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:06 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:06.870396 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:06.865330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:35933)
0806 22:38:06.865475 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:06.865490 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:06.865574 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:06.865760 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:06.865761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:06.865946 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:06.866002 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:06.866004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.866752 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.866753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:06.867277 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:06.867279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.867623 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.867623 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:06.867664 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:06.868232 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:06.868237 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:06.870330 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
W20260806 22:38:06.870543 30184 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:06.865415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:35933 (local address 127.16.235.126:37773)
0806 22:38:06.865594 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:06.865596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:06.865605 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:38:06.865630 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:06.865631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:06.865642 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:06.865702 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:06.866082 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.866596 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.867350 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.867545 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.867597 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:06.870431 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:35933: BlockingRecv error: recv got EOF from 127.16.235.65:35933 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
Aug 06 22:38:06 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:06 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:07.002700 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:06.997605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:47741)
0806 22:38:06.997808 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:06.997820 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:06.997879 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:06.998027 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:06.998028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:06.998213 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:06.998266 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:06.998268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.998944 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.998945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:06.999495 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:06.999497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.999825 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.999827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:06.999880 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:07.000363 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:07.000369 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:07.002565 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":133,"threads_started":1}
W20260806 22:38:07.002903 30184 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:06.997769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:47741 (local address 127.16.235.126:37773)
0806 22:38:06.997869 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:06.997871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:06.997887 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:38:06.997931 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:06.997932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:06.997944 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:06.998007 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:06.998337 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.998865 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.999573 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:06.999756 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:06.999805 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:07.002748 (+ 2943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:47741: BlockingRecv error: recv got EOF from 127.16.235.66:47741 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 22:38:07 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:07 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:07.158788 30186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:07.150335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:34193)
0806 22:38:07.150499 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:07.150510 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:07.150574 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:07.150754 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:07.150754 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:07.150948 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:07.151000 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:07.151001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:07.151645 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:07.151645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:07.152176 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:07.152178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:07.152564 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:07.152565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:07.152604 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:07.156448 (+ 3844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:07.156454 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:07.158717 (+ 2263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
W20260806 22:38:07.158942 30184 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:07.150402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:34193 (local address 127.16.235.126:37773)
0806 22:38:07.150476 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:07.150479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:07.150571 (+ 92us) server_negotiation.cc:407] Connection header received
0806 22:38:07.150619 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:07.150620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:07.150634 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:07.150745 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:07.151067 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:07.151576 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:07.152275 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:07.152487 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:07.152558 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:07.158854 (+ 6296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:34193: BlockingRecv error: recv got EOF from 127.16.235.67:34193 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 22:38:07 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:07 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:07.876379 30187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:07.871019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:33613)
0806 22:38:07.871173 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:07.871190 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:07.871244 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:07.871426 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:07.871427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:07.871611 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:07.871680 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:07.871683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:07.872373 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:07.872375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:07.872923 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:07.872926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:07.873273 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:07.873274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:07.873342 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:07.873912 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:07.873917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:07.876256 (+ 2339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
W20260806 22:38:07.876461 30188 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:07.871131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:33613 (local address 127.16.235.126:37773)
0806 22:38:07.871275 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:07.871277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:07.871286 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:38:07.871321 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:07.871321 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:07.871339 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:07.871409 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:07.871751 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:07.872264 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:07.873007 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:07.873197 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:07.873250 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:07.876371 (+ 3121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:33613: BlockingRecv error: recv got EOF from 127.16.235.65:33613 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
Aug 06 22:38:08 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:08 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:08.009197 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:08.003288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:54613)
0806 22:38:08.003529 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:08.003543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:08.003659 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:08.003895 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:08.003896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:08.004120 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:08.004173 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:08.004175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.005008 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.005009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:08.005623 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:08.005625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.006126 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.006127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:08.006208 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:08.006850 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:08.006856 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:08.009093 (+ 2237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":125,"threads_started":1}
W20260806 22:38:08.009318 30188 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:08.003406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:54613 (local address 127.16.235.126:37773)
0806 22:38:08.003525 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:08.003528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:08.003635 (+ 107us) server_negotiation.cc:407] Connection header received
0806 22:38:08.003783 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:08.003784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:08.003799 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:08.003886 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:08.004284 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.004893 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.005756 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.006056 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.006126 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:08.009236 (+ 3110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:54613: BlockingRecv error: recv got EOF from 127.16.235.66:54613 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 22:38:08 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:08 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:08.164801 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:08.159323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:46699)
0806 22:38:08.159472 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:08.159524 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:08.159597 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:08.159811 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:08.159812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:08.160015 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:08.160066 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:08.160068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.160759 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.160761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:08.161300 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:08.161302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.161677 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.161678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:08.161739 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:08.162274 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:08.162279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:08.164694 (+ 2415us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
W20260806 22:38:08.164867 30188 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:08.159486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:46699 (local address 127.16.235.126:37773)
0806 22:38:08.159566 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:08.159568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:08.159581 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:38:08.159678 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:08.159679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:08.159731 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:08.159802 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:08.160156 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.160662 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.161395 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.161591 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.161678 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:08.164792 (+ 3114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:46699: BlockingRecv error: recv got EOF from 127.16.235.67:46699 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 22:38:08 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:08 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:08.882931 30191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:08.877629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:56093)
0806 22:38:08.877840 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:08.877854 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:08.877927 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:08.878086 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:08.878086 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:08.878277 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:08.878329 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:08.878330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.879288 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.879289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:08.879870 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:08.879872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.880269 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.880270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:08.880310 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:08.880784 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:08.880788 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:08.882819 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":147,"threads_started":1}
W20260806 22:38:08.882979 30192 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:08.877756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:56093 (local address 127.16.235.126:37773)
0806 22:38:08.877867 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:08.877883 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:08.877911 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:38:08.877994 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:08.877995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:08.878007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:08.878068 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:08.878411 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.879201 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.879981 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:08.880190 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:08.880260 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:08.882904 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:56093: BlockingRecv error: recv got EOF from 127.16.235.65:56093 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":57,"threads_started":1}
Aug 06 22:38:09 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:09 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:09.015193 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:09.009896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:36497)
0806 22:38:09.010060 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:09.010075 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:09.010164 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:09.010319 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:09.010320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:09.010510 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:09.010561 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:09.010563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.011555 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.011556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:09.012143 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:09.012145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.012498 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.012499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:09.012540 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:09.013003 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:09.013008 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:09.015074 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
W20260806 22:38:09.015233 30192 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:09.010025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:36497 (local address 127.16.235.126:37773)
0806 22:38:09.010146 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:09.010148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:09.010160 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:38:09.010235 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:09.010236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:09.010248 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:09.010294 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:09.010636 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.011473 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.012225 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.012428 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.012484 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:09.015165 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:36497: BlockingRecv error: recv got EOF from 127.16.235.66:36497 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 22:38:09 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:09 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:09.170305 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:09.165361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:59265)
0806 22:38:09.165525 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:09.165536 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:09.165590 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:09.165741 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:09.165742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:09.165939 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:09.165989 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:09.165990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.166632 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.166633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:09.167221 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:09.167223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.167619 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.167619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:09.167659 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:09.168148 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:09.168152 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:09.170224 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
W20260806 22:38:09.170384 30192 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:09.165503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:59265 (local address 127.16.235.126:37773)
0806 22:38:09.165588 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:09.165591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:09.165635 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:38:09.165660 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:09.165661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:09.165673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:09.165716 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:09.166059 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.166558 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.167324 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.167540 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.167609 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:09.170332 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:59265: BlockingRecv error: recv got EOF from 127.16.235.67:59265 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 22:38:09 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:09 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:09.888320 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:09.883503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:48257)
0806 22:38:09.883666 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:09.883677 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:09.883731 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:09.883914 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:09.883915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:09.884101 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:09.884152 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:09.884153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.884841 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.884841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:09.885368 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:09.885370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.885739 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.885740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:09.885786 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:09.886273 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:09.886277 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:09.888257 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
W20260806 22:38:09.888496 30196 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:09.883627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:48257 (local address 127.16.235.126:37773)
0806 22:38:09.883763 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:09.883765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:09.883780 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:38:09.883806 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:09.883807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:09.883818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:09.883886 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:09.884234 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.884770 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.885448 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:09.885650 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:09.885714 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:09.888390 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:48257: BlockingRecv error: recv got EOF from 127.16.235.65:48257 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
Aug 06 22:38:10 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:10 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:10.020920 30199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:10.015764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:49033)
0806 22:38:10.016023 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:10.016036 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:10.016139 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:10.016293 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:10.016294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:10.016518 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:10.016565 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:10.016567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.017303 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.017304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:10.017864 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:10.017866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.018298 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.018300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:10.018362 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:10.019001 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:10.019006 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:10.020844 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":160,"threads_started":1}
W20260806 22:38:10.021024 30196 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:10.015887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:49033 (local address 127.16.235.126:37773)
0806 22:38:10.015991 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:10.015993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:10.016120 (+ 127us) server_negotiation.cc:407] Connection header received
0806 22:38:10.016214 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:10.016215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:10.016229 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:10.016286 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:10.016666 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.017204 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.017992 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.018201 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.018295 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:10.020953 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:49033: BlockingRecv error: recv got EOF from 127.16.235.66:49033 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 22:38:10 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:10 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:10.176366 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:10.171061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:43597)
0806 22:38:10.171246 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:10.171260 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:10.171334 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:10.171502 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:10.171502 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:10.171698 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:10.171745 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:10.171746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.172501 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.172501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:10.173135 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:10.173138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.173597 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.173598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:10.173640 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:10.174159 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:10.174163 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:10.176303 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
W20260806 22:38:10.176514 30196 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:10.171167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:43597 (local address 127.16.235.126:37773)
0806 22:38:10.171264 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:10.171268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:10.171338 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:38:10.171423 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:10.171424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:10.171439 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:10.171486 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:10.171828 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.172414 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.173266 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.173518 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.173585 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:10.176427 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:43597: BlockingRecv error: recv got EOF from 127.16.235.67:43597 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 22:38:10 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:10 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:10.893731 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:10.888869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:52115)
0806 22:38:10.889050 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:10.889061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:10.889115 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:10.889269 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:10.889270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:10.889463 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:10.889513 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:10.889516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.890175 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.890176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:10.890756 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:10.890757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.891100 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.891101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:10.891140 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:10.891613 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:10.891618 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:10.893615 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
W20260806 22:38:10.893780 30204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:10.888941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:52115 (local address 127.16.235.126:37773)
0806 22:38:10.889094 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:10.889096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:10.889105 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:38:10.889180 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:10.889180 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:10.889192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:10.889245 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:10.889595 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.890103 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.890827 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:10.891028 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:10.891086 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:10.893706 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:52115: BlockingRecv error: recv got EOF from 127.16.235.65:52115 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
Aug 06 22:38:11 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:11 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:11.026804 30205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.021584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:54827)
0806 22:38:11.021784 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:11.021799 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:11.021882 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:11.022034 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:11.022035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:11.022228 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:11.022282 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.022283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.023108 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.023109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.023673 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.023674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.024069 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.024070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.024114 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.024645 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:11.024650 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:11.026738 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
W20260806 22:38:11.026961 30204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:11.021759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:54827 (local address 127.16.235.126:37773)
0806 22:38:11.021837 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.021839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.021861 (+ 22us) server_negotiation.cc:407] Connection header received
0806 22:38:11.021952 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.021954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.021964 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.022011 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:11.022390 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.023023 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.023778 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.023990 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.024058 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.026875 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:54827: BlockingRecv error: recv got EOF from 127.16.235.66:54827 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 22:38:11 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:11 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:11.181632 30206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.176900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:53085)
0806 22:38:11.177054 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:11.177065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:11.177125 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:11.177247 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:11.177247 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:11.177441 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:11.177489 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.177491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.178147 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.178148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.178771 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.178774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.179171 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.179171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.179216 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.179738 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:11.179742 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:11.181559 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
W20260806 22:38:11.181797 30204 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:11.176976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:53085 (local address 127.16.235.126:37773)
0806 22:38:11.177027 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.177029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.177113 (+ 84us) server_negotiation.cc:407] Connection header received
0806 22:38:11.177172 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.177172 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.177186 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.177240 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:11.177558 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.178067 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.178885 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.179082 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.179150 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.181709 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:53085: BlockingRecv error: recv got EOF from 127.16.235.67:53085 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 06 22:38:11 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:11 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:11.899044 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:11.894276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:50923)
0806 22:38:11.894465 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:11.894476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:11.894518 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:11.894753 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:11.894754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:11.894940 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:11.894993 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.894994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.895685 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.895686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.896240 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:11.896242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.896587 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.896588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:11.896629 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.897134 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:11.897138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:11.898983 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":103,"threads_started":1}
W20260806 22:38:11.899166 30208 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:11.894399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:50923 (local address 127.16.235.126:37773)
0806 22:38:11.894543 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:11.894546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:11.894563 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:38:11.894587 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:11.894587 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:11.894598 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:11.894679 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:11.895065 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.895584 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.896322 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:11.896518 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:11.896570 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:11.899085 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:50923: BlockingRecv error: recv got EOF from 127.16.235.65:50923 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
Aug 06 22:38:12 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:12 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:12.032160 30209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:12.027357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:56381)
0806 22:38:12.027544 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:12.027563 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:12.027628 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:12.027777 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:12.027777 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:12.027983 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:12.028035 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:12.028036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.028682 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.028683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:12.029236 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:12.029237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.029592 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.029592 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:12.029639 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:12.030203 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:12.030208 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:12.032087 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":128,"threads_started":1}
W20260806 22:38:12.032287 30208 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:12.027419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:56381 (local address 127.16.235.126:37773)
0806 22:38:12.027496 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:12.027498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:12.027626 (+ 128us) server_negotiation.cc:407] Connection header received
0806 22:38:12.027694 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:12.027695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:12.027707 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:12.027749 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:12.028102 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.028607 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.029321 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.029516 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.029572 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:12.032206 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:56381: BlockingRecv error: recv got EOF from 127.16.235.66:56381 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 22:38:12 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:12 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.67@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:12.191794 30210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:12.182127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.67:49265)
0806 22:38:12.182304 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:12.182324 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:12.182393 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:12.182566 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:12.182567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:12.182853 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:12.182904 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:12.182906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.183604 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.183606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:12.184152 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:12.184154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.184533 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.184535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:12.184598 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:12.189604 (+ 5006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:12.189610 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:12.191733 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
W20260806 22:38:12.191946 30208 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:12.182278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:49265 (local address 127.16.235.126:37773)
0806 22:38:12.182372 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:12.182374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:12.182386 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:38:12.182467 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:12.182468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:12.182483 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:12.182557 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:12.183004 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.183514 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.184245 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.184452 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.184534 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:12.191860 (+ 7326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.67:49265: BlockingRecv error: recv got EOF from 127.16.235.67:49265 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 22:38:12 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:12 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.65@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:12.904202 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:12.899638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.65:38615)
0806 22:38:12.899806 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:12.899817 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:12.899869 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:12.900015 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:12.900015 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:12.900197 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:12.900251 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:12.900252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.900914 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.900915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:12.901458 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:12.901459 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.901784 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.901785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:12.901825 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:12.902305 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:12.902310 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:12.904129 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
W20260806 22:38:12.904315 30213 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:12.899712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:38615 (local address 127.16.235.126:37773)
0806 22:38:12.899870 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:12.899872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:12.899880 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:38:12.899920 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:12.899920 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:12.899930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:12.899991 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:12.900321 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.900841 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.901532 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:12.901719 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:12.901769 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:12.904238 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.65:38615: BlockingRecv error: recv got EOF from 127.16.235.65:38615 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
Aug 06 22:38:13 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:38:13 dist-test-slave-3kk6 krb5kdc[28705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.235.66@KRBTEST.COM for oryx/127.16.235.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:38:13.037781 30215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:13.032790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:37773 (local address 127.16.235.66:47039)
0806 22:38:13.033009 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:13.033022 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:13.033101 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:13.033274 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:13.033276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:13.033473 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:13.033525 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:13.033527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.034227 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.034228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:13.034851 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:13.034853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.035211 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.035212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:13.035271 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:13.035760 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:13.035765 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:13.037676 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.235.126:37773: Server oryx/127.16.235.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":117,"threads_started":1}
W20260806 22:38:13.037832 30213 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:13.032891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:47039 (local address 127.16.235.126:37773)
0806 22:38:13.032989 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:13.032994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:13.033079 (+ 85us) server_negotiation.cc:407] Connection header received
0806 22:38:13.033164 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:13.033165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:13.033181 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:13.033251 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:13.033615 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.034126 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.034936 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.035133 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.035212 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:13.037769 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.235.66:47039: BlockingRecv error: recv got EOF from 127.16.235.66:47039 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:38:13.083698 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 29528
I20260806 22:38:13.094579 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 29752
I20260806 22:38:13.100775 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 29941
I20260806 22:38:13.113449 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 29426
2026-08-06T22:38:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17804 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T22:38:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:38:13Z Disabled control of system clock
I20260806 22:38:13.149063 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:42037
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:42037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:38:13.235437 30222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:13.235723 30222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:13.235747 30222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:13.236954 30222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:38:13.236999 30222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:13.237012 30222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:13.237025 30222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:38:13.237038 30222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:38:13.238461 30222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:42037
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:42037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:13.238771 30222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:13.239061 30222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:13.244431 30231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.244451 30233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.244556 30230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:13.244451 30222 server_base.cc:1034] running on GCE node
I20260806 22:38:13.244984 30222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:13.245321 30222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:13.246496 30222 hybrid_clock.cc:648] HybridClock initialized: now 1786055893246487 us; error 38 us; skew 500 ppm
I20260806 22:38:13.247830 30222 webserver.cc:460] Webserver started at http://127.16.235.126:44439/ using document root <none> and password file <none>
I20260806 22:38:13.248164 30222 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:13.248237 30222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:13.248389 30222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:13.249533 30222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "874c22e1d8ca44aea0214c14da3830cd"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "627a052076502492d8886af6b8e5fb99"
server_key_iv: "cfbb5a7ff7adcac7425a1d34605a37cc"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.249935 30222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "874c22e1d8ca44aea0214c14da3830cd"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "627a052076502492d8886af6b8e5fb99"
server_key_iv: "cfbb5a7ff7adcac7425a1d34605a37cc"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.251391 30222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:38:13.252166 30243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.252339 30222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:38:13.252434 30222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "874c22e1d8ca44aea0214c14da3830cd"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "627a052076502492d8886af6b8e5fb99"
server_key_iv: "cfbb5a7ff7adcac7425a1d34605a37cc"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.252530 30222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:13.268970 30222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:13.269327 30222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:13.269505 30222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:13.273969 30222 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:42037
I20260806 22:38:13.274021 30323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:42037 every 8 connection(s)
I20260806 22:38:13.274413 30222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:38:13.275229 30324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:13.277861 30324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd: Bootstrap starting.
I20260806 22:38:13.278508 30324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:13.278863 30324 log.cc:824] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:13.279541 30324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd: No bootstrap required, opened a new log
I20260806 22:38:13.281082 30324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874c22e1d8ca44aea0214c14da3830cd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 42037 } }
I20260806 22:38:13.281231 30324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:13.281255 30324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874c22e1d8ca44aea0214c14da3830cd, State: Initialized, Role: FOLLOWER
I20260806 22:38:13.281410 30324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874c22e1d8ca44aea0214c14da3830cd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 42037 } }
I20260806 22:38:13.281500 30324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:38:13.281525 30324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:38:13.281554 30324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:13.282090 30324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874c22e1d8ca44aea0214c14da3830cd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 42037 } }
I20260806 22:38:13.282178 30324 leader_election.cc:302] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 874c22e1d8ca44aea0214c14da3830cd; no voters:
I20260806 22:38:13.282318 30324 leader_election.cc:288] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:38:13.282402 30332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:38:13.282598 30324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:38:13.282613 30332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [term 1 LEADER]: Becoming Leader. State: Replica: 874c22e1d8ca44aea0214c14da3830cd, State: Running, Role: LEADER
I20260806 22:38:13.282840 30332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874c22e1d8ca44aea0214c14da3830cd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 42037 } }
I20260806 22:38:13.283223 30333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "874c22e1d8ca44aea0214c14da3830cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874c22e1d8ca44aea0214c14da3830cd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 42037 } } }
I20260806 22:38:13.283396 30333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:13.283376 30335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 874c22e1d8ca44aea0214c14da3830cd. Latest consensus state: current_term: 1 leader_uuid: "874c22e1d8ca44aea0214c14da3830cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "874c22e1d8ca44aea0214c14da3830cd" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 42037 } } }
I20260806 22:38:13.283543 30335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:13.283897 30343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:38:13.284451 30343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:38:13.284538 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 30222
I20260806 22:38:13.284639 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:38:13.284840 17325 external_mini_cluster.cc:1428] Setting key 48502f0a5c7a0eb8f2a240dc92cfd1b3
I20260806 22:38:13.286751 30343 catalog_manager.cc:1347] Generated new cluster ID: 33872df474d5493398f27a9da86879ad
I20260806 22:38:13.286809 30343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:38:13.291265 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:13.285364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.16.235.126:42037)
0806 22:38:13.285565 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:13.285575 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:13.285609 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:38:13.285728 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:13.285735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:13.285803 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:13.285957 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:13.286202 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.287618 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.288693 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.289302 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.289414 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:13.290379 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:13.290383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:13.290384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:38:13.290607 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:13.290668 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:38:13.290684 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:13.290737 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:13.290989 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 22:38:13.291064 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260806 22:38:13.295157 30343 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:38:13.295661 30343 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:38:13.298388 30343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 874c22e1d8ca44aea0214c14da3830cd: Generated new TSK 0
I20260806 22:38:13.298566 30343 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:38:13.299803 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:42037
--builtin_ntp_servers=127.16.235.84:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:38:13.387836 30366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:13.388104 30366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:13.388144 30366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:13.389397 30366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:13.389465 30366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:13.389505 30366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:38:13.390969 30366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:42037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:13.391245 30366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:13.391573 30366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:13.397718 30373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.397802 30377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.397688 30372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:13.398111 30366 server_base.cc:1034] running on GCE node
I20260806 22:38:13.398350 30366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:13.398685 30366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:13.399883 30366 hybrid_clock.cc:648] HybridClock initialized: now 1786055893399843 us; error 54 us; skew 500 ppm
I20260806 22:38:13.401554 30366 webserver.cc:460] Webserver started at http://127.16.235.65:43219/ using document root <none> and password file <none>
I20260806 22:38:13.401844 30366 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:13.401921 30366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:13.402068 30366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:13.403569 30366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "15e7c7046775434091c3d3ca61dd9048"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "735d3ae39fb540e657ea7455c661d20e"
server_key_iv: "56804d95a05ba3a82a105d0927087c24"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.403992 30366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "15e7c7046775434091c3d3ca61dd9048"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "735d3ae39fb540e657ea7455c661d20e"
server_key_iv: "56804d95a05ba3a82a105d0927087c24"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.405679 30366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:38:13.406792 30390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.407092 30366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:38:13.407200 30366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "15e7c7046775434091c3d3ca61dd9048"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "735d3ae39fb540e657ea7455c661d20e"
server_key_iv: "56804d95a05ba3a82a105d0927087c24"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.407275 30366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:13.421598 30366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:13.421968 30366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:13.422153 30366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:13.422446 30366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:13.423000 30366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:13.423058 30366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.423094 30366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:13.423120 30366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.430299 30366 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:34651
I20260806 22:38:13.430342 30555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:34651 every 8 connection(s)
I20260806 22:38:13.430805 30366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:38:13.436173 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 30366
I20260806 22:38:13.436257 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:38:13.436460 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:13.431669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:33319 (local address 127.16.235.126:42037)
0806 22:38:13.431756 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:13.431758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:13.432033 (+ 275us) server_negotiation.cc:407] Connection header received
0806 22:38:13.432563 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:13.432565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:13.432578 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:13.432646 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:13.433372 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.433897 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.434964 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.435180 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.435261 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:13.436085 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:13.436087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:13.436087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:38:13.436095 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:13.436111 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:38:13.436116 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:13.436171 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:13.436333 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:38:13.436368 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:38:13.436508 17325 external_mini_cluster.cc:1428] Setting key 597710c9b59f6acc7dc05e7fec4bf824
I20260806 22:38:13.436640 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:13.431610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:42037 (local address 127.16.235.65:33319)
0806 22:38:13.431867 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:13.431918 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:13.432444 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:13.432725 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:13.432729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:13.432741 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:13.433263 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:13.433267 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.434010 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.434011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:13.434849 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:13.434851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.435270 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.435271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:13.435412 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:13.435937 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:38:13.435937 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:13.435966 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:38:13.435989 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:13.436187 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:13.436193 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:13.436196 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:38:13.436293 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:38:13.436414 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":121,"threads_started":1}
I20260806 22:38:13.437111 30558 heartbeater.cc:347] Connected to a master server at 127.16.235.126:42037
I20260806 22:38:13.437234 30558 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:13.437459 30558 heartbeater.cc:511] Master 127.16.235.126:42037 requested a full tablet report, sending...
I20260806 22:38:13.437618 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:42037
--builtin_ntp_servers=127.16.235.84:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:38:13.438076 30271 ts_manager.cc:194] Registered new tserver with Master: 15e7c7046775434091c3d3ca61dd9048 (127.16.235.65:34651)
I20260806 22:38:13.438999 30271 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.65:33319
W20260806 22:38:13.523134 30562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:13.523392 30562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:13.523433 30562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:13.524576 30562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:13.524619 30562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:13.524644 30562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:38:13.525938 30562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:42037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:13.526139 30562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:13.526409 30562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:13.531567 30570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.531600 30573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.531567 30571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:13.531829 30562 server_base.cc:1034] running on GCE node
I20260806 22:38:13.532059 30562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:13.532320 30562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:13.533488 30562 hybrid_clock.cc:648] HybridClock initialized: now 1786055893533474 us; error 31 us; skew 500 ppm
I20260806 22:38:13.534842 30562 webserver.cc:460] Webserver started at http://127.16.235.66:35991/ using document root <none> and password file <none>
I20260806 22:38:13.535104 30562 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:13.535178 30562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:13.535347 30562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:13.536504 30562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "be95530ae42a40c3bf794827ca025fe9"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "96d3ead3e7370ec45291d2b5dc688368"
server_key_iv: "0b50f39806dd464f433f5627beb1f7e4"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.536896 30562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "be95530ae42a40c3bf794827ca025fe9"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "96d3ead3e7370ec45291d2b5dc688368"
server_key_iv: "0b50f39806dd464f433f5627beb1f7e4"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.538275 30562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:38:13.539114 30580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.539291 30562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.539402 30562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "be95530ae42a40c3bf794827ca025fe9"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "96d3ead3e7370ec45291d2b5dc688368"
server_key_iv: "0b50f39806dd464f433f5627beb1f7e4"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.539481 30562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:13.566938 30562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:13.567319 30562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:13.567488 30562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:13.567752 30562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:13.568126 30562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:13.568183 30562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.568228 30562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:13.568254 30562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.574517 30562 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:36217
I20260806 22:38:13.574574 30754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:36217 every 8 connection(s)
I20260806 22:38:13.574970 30562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:38:13.580560 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:13.575843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:56863 (local address 127.16.235.126:42037)
0806 22:38:13.575946 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:13.575948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:13.576212 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:38:13.576700 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:13.576702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:13.576716 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:13.576785 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:13.577467 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.577987 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.579139 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.579338 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.579407 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:13.580278 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:13.580280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:13.580280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:38:13.580285 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:13.580301 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:38:13.580305 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:13.580361 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:13.580461 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:38:13.580494 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:38:13.580725 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:13.575817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:42037 (local address 127.16.235.66:56863)
0806 22:38:13.576023 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:13.576068 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:13.576586 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:13.576846 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:13.576850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:13.576859 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:13.577382 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:13.577387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.578079 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.578080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:13.579042 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:13.579046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.579422 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.579422 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:13.579535 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:13.580113 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:38:13.580113 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:13.580144 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:38:13.580169 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:13.580371 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:13.580375 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:13.580378 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:38:13.580433 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:38:13.580520 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260806 22:38:13.581308 30755 heartbeater.cc:347] Connected to a master server at 127.16.235.126:42037
I20260806 22:38:13.581427 30755 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:13.581625 30755 heartbeater.cc:511] Master 127.16.235.126:42037 requested a full tablet report, sending...
I20260806 22:38:13.582098 30271 ts_manager.cc:194] Registered new tserver with Master: be95530ae42a40c3bf794827ca025fe9 (127.16.235.66:36217)
I20260806 22:38:13.582538 30271 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.66:56863
I20260806 22:38:13.584508 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 30562
I20260806 22:38:13.584625 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:38:13.584811 17325 external_mini_cluster.cc:1428] Setting key bcf9c0f9cd1d24ee78bbf89ff642a942
I20260806 22:38:13.585744 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:42037
--builtin_ntp_servers=127.16.235.84:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:38:13.670127 30760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:13.670356 30760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:13.670377 30760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:13.671590 30760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:13.671634 30760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:13.671676 30760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:38:13.673033 30760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:42037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:13.673288 30760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:13.673611 30760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:13.678763 30770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.678788 30767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.678757 30768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:13.679126 30760 server_base.cc:1034] running on GCE node
I20260806 22:38:13.679342 30760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:13.679543 30760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:13.680678 30760 hybrid_clock.cc:648] HybridClock initialized: now 1786055893680647 us; error 51 us; skew 500 ppm
I20260806 22:38:13.681962 30760 webserver.cc:460] Webserver started at http://127.16.235.67:34799/ using document root <none> and password file <none>
I20260806 22:38:13.682154 30760 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:13.682195 30760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:13.682291 30760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:13.683471 30760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "4978b2eb48d94db290ebace0ce671c4f"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "2feee4bd3d34a3c88ebeb72d38811112"
server_key_iv: "6d9b66390158e49a18356423a7bcac40"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.683830 30760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "4978b2eb48d94db290ebace0ce671c4f"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "2feee4bd3d34a3c88ebeb72d38811112"
server_key_iv: "6d9b66390158e49a18356423a7bcac40"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.685206 30760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:38:13.685916 30777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.686107 30760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:38:13.686182 30760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "4978b2eb48d94db290ebace0ce671c4f"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "2feee4bd3d34a3c88ebeb72d38811112"
server_key_iv: "6d9b66390158e49a18356423a7bcac40"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.686236 30760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:13.697535 30760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:13.697809 30760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:13.697919 30760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:13.698105 30760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:13.698475 30760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:13.698513 30760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.698534 30760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:13.698582 30760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.704787 30760 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:38853
I20260806 22:38:13.704852 30963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:38853 every 8 connection(s)
I20260806 22:38:13.705195 30760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:38:13.710572 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:13.706086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:41659 (local address 127.16.235.126:42037)
0806 22:38:13.706184 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:13.706187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:13.706355 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:38:13.706966 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:13.706967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:13.706980 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:13.707050 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:13.707776 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.708286 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.709243 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.709424 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.709477 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:13.710359 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:13.710361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:13.710361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:38:13.710365 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:13.710375 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:38:13.710379 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:13.710417 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:13.710491 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:38:13.710511 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:38:13.710915 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:13.705997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:42037 (local address 127.16.235.67:41659)
0806 22:38:13.706199 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:13.706236 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:13.706832 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:13.707154 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:13.707158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:13.707166 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:13.707683 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:13.707687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.708375 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.708376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:13.709169 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:13.709171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:13.709501 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:13.709501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:13.709610 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:13.710235 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:38:13.710236 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:13.710265 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:38:13.710288 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:13.710430 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:13.710432 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:13.710434 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:38:13.710482 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:38:13.710560 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260806 22:38:13.711480 30966 heartbeater.cc:347] Connected to a master server at 127.16.235.126:42037
I20260806 22:38:13.711596 30966 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:13.711617 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 30760
I20260806 22:38:13.711732 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:38:13.711824 30966 heartbeater.cc:511] Master 127.16.235.126:42037 requested a full tablet report, sending...
I20260806 22:38:13.711993 17325 external_mini_cluster.cc:1428] Setting key 05c4ce97171e89e2a4949d0712ab3b38
I20260806 22:38:13.712253 30271 ts_manager.cc:194] Registered new tserver with Master: 4978b2eb48d94db290ebace0ce671c4f (127.16.235.67:38853)
I20260806 22:38:13.712744 30271 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.67:41659
I20260806 22:38:13.712970 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:38:13.714944 30364 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:13.714415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.16.235.126:42037)
0806 22:38:13.714498 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:13.714500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:13.714595 (+ 95us) server_negotiation.cc:407] Connection header received
0806 22:38:13.714634 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:13.714634 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:13.714643 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:13.714744 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:13.714878 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43736: BlockingRecv error: recv got EOF from 127.0.0.1:43736 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:38:13.715006 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 30366
I20260806 22:38:13.734855 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 30562
I20260806 22:38:13.751605 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 30760
I20260806 22:38:13.761881 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 30222
2026-08-06T22:38:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (650 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T22:38:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:38:13Z Disabled control of system clock
I20260806 22:38:13.800588 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:41669
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:41145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:41669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:38:13.886296 30982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:13.886519 30982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:13.886543 30982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:13.887692 30982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:38:13.887737 30982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:13.887750 30982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:13.887763 30982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:38:13.887775 30982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:38:13.889067 30982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:41145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:41669
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:41669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:13.889271 30982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:13.889550 30982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:13.894573 30991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.894579 30989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:13.894562 30988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:13.895036 30982 server_base.cc:1034] running on GCE node
I20260806 22:38:13.895324 30982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:13.895628 30982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:13.896816 30982 hybrid_clock.cc:648] HybridClock initialized: now 1786055893896794 us; error 49 us; skew 500 ppm
I20260806 22:38:13.898165 30982 webserver.cc:460] Webserver started at http://127.16.235.126:37035/ using document root <none> and password file <none>
I20260806 22:38:13.898451 30982 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:13.898525 30982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:13.898746 30982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:13.899920 30982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "5180e722c6a82f0853801368bd9c2b24"
server_key_iv: "cff6e0d0022c16722ff31561d9cb894e"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.900319 30982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "5180e722c6a82f0853801368bd9c2b24"
server_key_iv: "cff6e0d0022c16722ff31561d9cb894e"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.901710 30982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:38:13.902482 31002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:13.902720 30982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:38:13.902848 30982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96"
format_stamp: "Formatted at 2026-08-06 22:38:13 on dist-test-slave-3kk6"
server_key: "5180e722c6a82f0853801368bd9c2b24"
server_key_iv: "cff6e0d0022c16722ff31561d9cb894e"
server_key_version: "encryptionkey@0"
I20260806 22:38:13.902931 30982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:13.906533 30982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:13.906874 30982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:13.907033 30982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:13.911108 30982 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:41669
I20260806 22:38:13.911142 31090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:41669 every 8 connection(s)
I20260806 22:38:13.911484 30982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:38:13.912204 31093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:13.915227 31093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96: Bootstrap starting.
I20260806 22:38:13.915455 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 30982
I20260806 22:38:13.915560 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:38:13.915751 17325 external_mini_cluster.cc:1428] Setting key 7baacd08ec82052279aa394297b6010e
I20260806 22:38:13.915990 31093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:13.916374 31093 log.cc:824] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:13.917096 31093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96: No bootstrap required, opened a new log
I20260806 22:38:13.918609 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:13.916464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41382 (local address 127.16.235.126:41669)
0806 22:38:13.916671 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:13.916679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:13.916704 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:38:13.917155 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:13.917164 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:13.917221 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:13.917377 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:13.918157 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:13.918161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:13.918161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:38:13.918166 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:13.918208 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:38:13.918218 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:13.918314 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:13.918375 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:38:13.918435 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260806 22:38:13.919054 31093 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 41669 } }
I20260806 22:38:13.919234 31093 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:13.919273 31093 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1dfa4a38554ba3b8ae5767ed4f0d96, State: Initialized, Role: FOLLOWER
I20260806 22:38:13.919392 31093 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 41669 } }
I20260806 22:38:13.919485 31093 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:38:13.919525 31093 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:38:13.919581 31093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:13.920469 31093 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 41669 } }
I20260806 22:38:13.920598 31093 leader_election.cc:302] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad1dfa4a38554ba3b8ae5767ed4f0d96; no voters:
I20260806 22:38:13.920790 31093 leader_election.cc:288] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:38:13.920902 31099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:38:13.921046 31099 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [term 1 LEADER]: Becoming Leader. State: Replica: ad1dfa4a38554ba3b8ae5767ed4f0d96, State: Running, Role: LEADER
I20260806 22:38:13.921103 31093 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:38:13.921146 31099 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 41669 } }
I20260806 22:38:13.921561 31100 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 41669 } } }
I20260806 22:38:13.921672 31100 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:13.921900 31107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:38:13.922112 31101 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1dfa4a38554ba3b8ae5767ed4f0d96. Latest consensus state: current_term: 1 leader_uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1dfa4a38554ba3b8ae5767ed4f0d96" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 41669 } } }
I20260806 22:38:13.922204 31101 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96 [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:13.922428 31107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:38:13.924021 31107 catalog_manager.cc:1347] Generated new cluster ID: 21c6d0b4bc0a4e6fa850509c4497944b
I20260806 22:38:13.924074 31107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:38:13.952046 31107 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:38:13.952302 31107 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:38:13.957317 31107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad1dfa4a38554ba3b8ae5767ed4f0d96: Generated new TSK 0
I20260806 22:38:13.957549 31107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:38:13.969182 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:41669
--builtin_ntp_servers=127.16.235.84:41145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:38:14.058058 31126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:14.058300 31126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:14.058324 31126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:14.059573 31126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:14.059621 31126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:14.059650 31126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:38:14.061203 31126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:41145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:14.061478 31126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:14.061811 31126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:14.066984 31136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:14.067085 31132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:14.067211 31134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:14.067211 31126 server_base.cc:1034] running on GCE node
I20260806 22:38:14.067497 31126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:14.067764 31126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:14.068924 31126 hybrid_clock.cc:648] HybridClock initialized: now 1786055894068908 us; error 37 us; skew 500 ppm
I20260806 22:38:14.070394 31126 webserver.cc:460] Webserver started at http://127.16.235.65:33825/ using document root <none> and password file <none>
I20260806 22:38:14.070698 31126 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:14.070765 31126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:14.070926 31126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:14.072086 31126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "9799c60376fa4772ba4947863b25ff59"
format_stamp: "Formatted at 2026-08-06 22:38:14 on dist-test-slave-3kk6"
server_key: "a0a55000d2a22562d37a12149cc78742"
server_key_iv: "5c7f0a5dfdb9d3bea6418ab91e246be7"
server_key_version: "encryptionkey@0"
I20260806 22:38:14.072472 31126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "9799c60376fa4772ba4947863b25ff59"
format_stamp: "Formatted at 2026-08-06 22:38:14 on dist-test-slave-3kk6"
server_key: "a0a55000d2a22562d37a12149cc78742"
server_key_iv: "5c7f0a5dfdb9d3bea6418ab91e246be7"
server_key_version: "encryptionkey@0"
I20260806 22:38:14.073866 31126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:38:14.074707 31147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:14.074901 31126 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:38:14.074992 31126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "9799c60376fa4772ba4947863b25ff59"
format_stamp: "Formatted at 2026-08-06 22:38:14 on dist-test-slave-3kk6"
server_key: "a0a55000d2a22562d37a12149cc78742"
server_key_iv: "5c7f0a5dfdb9d3bea6418ab91e246be7"
server_key_version: "encryptionkey@0"
I20260806 22:38:14.075050 31126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:14.078409 31126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:14.078692 31126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:14.078816 31126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:14.078990 31126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:14.079264 31126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:14.079296 31126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:14.079317 31126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:14.079330 31126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:14.085304 31126 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:41515
I20260806 22:38:14.085382 31304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:41515 every 8 connection(s)
I20260806 22:38:14.085727 31126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:38:14.085883 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 31126
I20260806 22:38:14.086002 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:38:14.086221 17325 external_mini_cluster.cc:1428] Setting key 8a8f7a2af8880f48f950383eb6edad68
I20260806 22:38:14.087575 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:41669
--builtin_ntp_servers=127.16.235.84:41145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 22:38:14.089581 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:14.086578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:38061 (local address 127.16.235.126:41669)
0806 22:38:14.086623 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:14.086626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:14.086937 (+ 311us) server_negotiation.cc:407] Connection header received
0806 22:38:14.087686 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:14.087687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:14.087692 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:14.087736 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:14.089272 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:14.089274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:14.089274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:38:14.089279 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:14.089297 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:38:14.089302 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:14.089362 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:14.089495 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:38:14.089511 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:38:14.089883 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:14.086576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:41669 (local address 127.16.235.65:38061)
0806 22:38:14.086814 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:14.086849 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:14.087559 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:14.087811 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:14.087816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:14.087826 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:14.089012 (+ 1186us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:38:14.089013 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:14.089054 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:38:14.089094 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:14.089395 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:14.089405 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:14.089408 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:38:14.089495 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:38:14.089566 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":133,"threads_started":1}
I20260806 22:38:14.090390 31305 heartbeater.cc:347] Connected to a master server at 127.16.235.126:41669
I20260806 22:38:14.090502 31305 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:14.090745 31305 heartbeater.cc:511] Master 127.16.235.126:41669 requested a full tablet report, sending...
I20260806 22:38:14.091323 31030 ts_manager.cc:194] Registered new tserver with Master: 9799c60376fa4772ba4947863b25ff59 (127.16.235.65:41515)
W20260806 22:38:14.178050 31311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:14.178287 31311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:14.178318 31311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:14.179618 31311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:14.179672 31311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:14.179698 31311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:38:14.181169 31311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:41145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:14.181437 31311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:14.181779 31311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:14.187109 31322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:14.187079 31320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:14.187263 31319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:14.187531 31311 server_base.cc:1034] running on GCE node
I20260806 22:38:14.187788 31311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:14.188023 31311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:14.189208 31311 hybrid_clock.cc:648] HybridClock initialized: now 1786055894189190 us; error 31 us; skew 500 ppm
I20260806 22:38:14.191018 31311 webserver.cc:460] Webserver started at http://127.16.235.66:40811/ using document root <none> and password file <none>
I20260806 22:38:14.191274 31311 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:14.191325 31311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:14.191433 31311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:14.192647 31311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "3a7e73bad8fe4a85ad4ac17cce5838a3"
format_stamp: "Formatted at 2026-08-06 22:38:14 on dist-test-slave-3kk6"
server_key: "4a1db1c8a26e4557c77cd97d0769ba30"
server_key_iv: "364514fddf66ae3ad21552a730024c71"
server_key_version: "encryptionkey@0"
I20260806 22:38:14.193023 31311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "3a7e73bad8fe4a85ad4ac17cce5838a3"
format_stamp: "Formatted at 2026-08-06 22:38:14 on dist-test-slave-3kk6"
server_key: "4a1db1c8a26e4557c77cd97d0769ba30"
server_key_iv: "364514fddf66ae3ad21552a730024c71"
server_key_version: "encryptionkey@0"
I20260806 22:38:14.194548 31311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:38:14.195542 31335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:14.195761 31311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:38:14.195851 31311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "3a7e73bad8fe4a85ad4ac17cce5838a3"
format_stamp: "Formatted at 2026-08-06 22:38:14 on dist-test-slave-3kk6"
server_key: "4a1db1c8a26e4557c77cd97d0769ba30"
server_key_iv: "364514fddf66ae3ad21552a730024c71"
server_key_version: "encryptionkey@0"
I20260806 22:38:14.195930 31311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:14.199697 31311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:14.199952 31311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:14.200064 31311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:14.200253 31311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:14.200536 31311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:14.200570 31311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:14.200593 31311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:14.200608 31311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:14.207307 31311 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:42533
I20260806 22:38:14.207440 31505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:42533 every 8 connection(s)
I20260806 22:38:14.207769 31311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:38:14.211179 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:14.208663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:34085 (local address 127.16.235.126:41669)
0806 22:38:14.208737 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:14.208739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:14.208962 (+ 223us) server_negotiation.cc:407] Connection header received
0806 22:38:14.209502 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:14.209504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:14.209510 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:14.209582 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:14.210941 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:14.210943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:14.210943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:38:14.210948 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:14.210964 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:38:14.210969 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:14.211036 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:14.211104 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 22:38:14.211117 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:38:14.211436 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:14.208614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:41669 (local address 127.16.235.66:34085)
0806 22:38:14.208816 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:14.208861 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:14.209374 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:14.209669 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:14.209672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:14.209681 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:14.210752 (+ 1071us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:38:14.210753 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:14.210795 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:38:14.210829 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:14.211046 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:14.211051 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:14.211054 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:38:14.211116 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:38:14.211174 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260806 22:38:14.212116 31506 heartbeater.cc:347] Connected to a master server at 127.16.235.126:41669
I20260806 22:38:14.212237 31506 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:14.212402 31506 heartbeater.cc:511] Master 127.16.235.126:41669 requested a full tablet report, sending...
I20260806 22:38:14.212903 31030 ts_manager.cc:194] Registered new tserver with Master: 3a7e73bad8fe4a85ad4ac17cce5838a3 (127.16.235.66:42533)
I20260806 22:38:14.214778 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 31311
I20260806 22:38:14.214877 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:38:14.215045 17325 external_mini_cluster.cc:1428] Setting key 60379be288446f7ded56f3572d43901a
I20260806 22:38:14.216096 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:41669
--builtin_ntp_servers=127.16.235.84:41145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:38:14.306634 31509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:14.307040 31509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:14.307070 31509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:14.308450 31509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:14.308508 31509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:14.308538 31509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:38:14.310091 31509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:41145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:14.310382 31509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:14.310797 31509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:14.316195 31524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:14.316182 31518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:14.316473 31509 server_base.cc:1034] running on GCE node
W20260806 22:38:14.316561 31519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:14.316793 31509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:14.317018 31509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:14.318205 31509 hybrid_clock.cc:648] HybridClock initialized: now 1786055894318199 us; error 36 us; skew 500 ppm
I20260806 22:38:14.319693 31509 webserver.cc:460] Webserver started at http://127.16.235.67:36657/ using document root <none> and password file <none>
I20260806 22:38:14.319996 31509 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:14.320077 31509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:14.320240 31509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:14.321449 31509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "72821c603063486689019cbd2bd03124"
format_stamp: "Formatted at 2026-08-06 22:38:14 on dist-test-slave-3kk6"
server_key: "b84f799b973872dbc20ccb6062572461"
server_key_iv: "bc271265515497000e1e553663e71189"
server_key_version: "encryptionkey@0"
I20260806 22:38:14.321866 31509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "72821c603063486689019cbd2bd03124"
format_stamp: "Formatted at 2026-08-06 22:38:14 on dist-test-slave-3kk6"
server_key: "b84f799b973872dbc20ccb6062572461"
server_key_iv: "bc271265515497000e1e553663e71189"
server_key_version: "encryptionkey@0"
I20260806 22:38:14.323343 31509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:38:14.324127 31533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:14.324334 31509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:38:14.324434 31509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "72821c603063486689019cbd2bd03124"
format_stamp: "Formatted at 2026-08-06 22:38:14 on dist-test-slave-3kk6"
server_key: "b84f799b973872dbc20ccb6062572461"
server_key_iv: "bc271265515497000e1e553663e71189"
server_key_version: "encryptionkey@0"
I20260806 22:38:14.324524 31509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:14.328171 31509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:14.328467 31509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:14.328624 31509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:14.328866 31509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:14.329195 31509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:14.329250 31509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:14.329291 31509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:14.329319 31509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:14.335816 31509 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:35847
I20260806 22:38:14.335856 31706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:35847 every 8 connection(s)
I20260806 22:38:14.336241 31509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:38:14.339586 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:14.337203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:39799 (local address 127.16.235.126:41669)
0806 22:38:14.337296 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:14.337299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:14.337555 (+ 256us) server_negotiation.cc:407] Connection header received
0806 22:38:14.338067 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:14.338068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:14.338074 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:14.338133 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:14.339348 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:14.339350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:14.339350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:38:14.339355 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:14.339372 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:38:14.339376 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:14.339425 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:14.339514 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:38:14.339527 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:38:14.339731 31709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:14.337200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:41669 (local address 127.16.235.67:39799)
0806 22:38:14.337429 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:14.337467 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:14.337969 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:14.338191 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:14.338195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:14.338203 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:14.339195 (+ 992us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:38:14.339196 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:14.339227 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:38:14.339257 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:14.339423 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:14.339425 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:14.339428 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:38:14.339482 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:38:14.339535 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":126,"threads_started":1}
I20260806 22:38:14.340309 31707 heartbeater.cc:347] Connected to a master server at 127.16.235.126:41669
I20260806 22:38:14.340451 31707 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:14.340646 31707 heartbeater.cc:511] Master 127.16.235.126:41669 requested a full tablet report, sending...
I20260806 22:38:14.341114 31030 ts_manager.cc:194] Registered new tserver with Master: 72821c603063486689019cbd2bd03124 (127.16.235.67:35847)
I20260806 22:38:14.342489 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 31509
I20260806 22:38:14.342569 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:38:14.342864 17325 external_mini_cluster.cc:1428] Setting key 926553b1bd1258f1e826e14a487d0e4b
I20260806 22:38:14.343679 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:38:14.345678 31097 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:38:14.345059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.16.235.126:41669)
0806 22:38:14.345130 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:14.345132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:14.345268 (+ 136us) server_negotiation.cc:407] Connection header received
0806 22:38:14.345351 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:14.345352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:14.345360 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:14.345430 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:14.345601 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41396: BlockingRecv error: recv got EOF from 127.0.0.1:41396 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:38:14.345711 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 31126
I20260806 22:38:14.359041 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 31311
I20260806 22:38:14.374617 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 31509
I20260806 22:38:14.389396 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 30982
2026-08-06T22:38:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (624 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:38:14 dist-test-slave-3kk6 krb5kdc[31718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:38:14 dist-test-slave-3kk6 krb5kdc[31718](info): set up 2 sockets
Aug 06 22:38:14 dist-test-slave-3kk6 krb5kdc[31718](info): commencing operation
krb5kdc: starting...
W20260806 22:38:15.623885 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:38:15 dist-test-slave-3kk6 krb5kdc[31718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:38:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:38:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:38:15.761358 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33525
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:33525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:38:15.848497 31736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:15.848766 31736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:15.848810 31736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:15.850003 31736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:38:15.850078 31736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:15.850103 31736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:15.850137 31736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:38:15.850165 31736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:38:15.851828 31736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:33525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:15.852133 31736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:15.852383 31736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:15.857636 31747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:15.857623 31745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:15.858059 31736 server_base.cc:1034] running on GCE node
W20260806 22:38:15.858070 31749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:15.858475 31736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:15.858911 31736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:15.860126 31736 hybrid_clock.cc:648] HybridClock initialized: now 1786055895860100 us; error 36 us; skew 500 ppm
Aug 06 22:38:15 dist-test-slave-3kk6 krb5kdc[31718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055895, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:15.862843 31736 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:38:15.863304 31736 webserver.cc:460] Webserver started at http://127.16.235.126:32823/ using document root <none> and password file <none>
I20260806 22:38:15.863551 31736 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:15.863620 31736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:15.863765 31736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:15.864916 31736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "974ee7e2815c48c8be751b08bbdca697"
format_stamp: "Formatted at 2026-08-06 22:38:15 on dist-test-slave-3kk6"
server_key: "9ea189f1df2f9f421d3e49c50de90aee"
server_key_iv: "3108932a9037b598d1a914a1acdf3622"
server_key_version: "encryptionkey@0"
I20260806 22:38:15.865316 31736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "974ee7e2815c48c8be751b08bbdca697"
format_stamp: "Formatted at 2026-08-06 22:38:15 on dist-test-slave-3kk6"
server_key: "9ea189f1df2f9f421d3e49c50de90aee"
server_key_iv: "3108932a9037b598d1a914a1acdf3622"
server_key_version: "encryptionkey@0"
I20260806 22:38:15.866791 31736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:38:15.867725 31757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:15.867908 31736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:38:15.868017 31736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "974ee7e2815c48c8be751b08bbdca697"
format_stamp: "Formatted at 2026-08-06 22:38:15 on dist-test-slave-3kk6"
server_key: "9ea189f1df2f9f421d3e49c50de90aee"
server_key_iv: "3108932a9037b598d1a914a1acdf3622"
server_key_version: "encryptionkey@0"
I20260806 22:38:15.868098 31736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:15.883996 31736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:15.886268 31736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:15.886459 31736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:15.891013 31736 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:33525
I20260806 22:38:15.891058 31838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:33525 every 8 connection(s)
I20260806 22:38:15.891481 31736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:38:15.892529 31840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:15.895068 31840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697: Bootstrap starting.
I20260806 22:38:15.895701 31840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:15.896034 31840 log.cc:824] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:15.896788 31840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697: No bootstrap required, opened a new log
I20260806 22:38:15.897753 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 31736
I20260806 22:38:15.897862 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:38:15.898061 17325 external_mini_cluster.cc:1428] Setting key b48ba3dbf505b568371463ef27c320c4
I20260806 22:38:15.898717 31840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ee7e2815c48c8be751b08bbdca697" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33525 } }
I20260806 22:38:15.898942 31840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:15.898998 31840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974ee7e2815c48c8be751b08bbdca697, State: Initialized, Role: FOLLOWER
I20260806 22:38:15.899139 31840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ee7e2815c48c8be751b08bbdca697" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33525 } }
I20260806 22:38:15.899227 31840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:38:15.899269 31840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:38:15.899322 31840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:15.900333 31840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ee7e2815c48c8be751b08bbdca697" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33525 } }
I20260806 22:38:15.900449 31840 leader_election.cc:302] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 974ee7e2815c48c8be751b08bbdca697; no voters:
I20260806 22:38:15.900614 31840 leader_election.cc:288] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:38:15.900673 31847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:38:15.900828 31847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [term 1 LEADER]: Becoming Leader. State: Replica: 974ee7e2815c48c8be751b08bbdca697, State: Running, Role: LEADER
I20260806 22:38:15.900866 31840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:38:15.900914 31847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ee7e2815c48c8be751b08bbdca697" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33525 } }
I20260806 22:38:15.901237 31847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 974ee7e2815c48c8be751b08bbdca697. Latest consensus state: current_term: 1 leader_uuid: "974ee7e2815c48c8be751b08bbdca697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ee7e2815c48c8be751b08bbdca697" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33525 } } }
I20260806 22:38:15.901327 31847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:15.901248 31848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "974ee7e2815c48c8be751b08bbdca697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ee7e2815c48c8be751b08bbdca697" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33525 } } }
I20260806 22:38:15.901486 31848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697 [sys.catalog]: This master's current role is: LEADER
I20260806 22:38:15.901741 31855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:38:15.902343 31855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:38:15.904446 31855 catalog_manager.cc:1347] Generated new cluster ID: c26bc05f04ab4e6f8a2d0d529bea9398
I20260806 22:38:15.904520 31855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:38:15 dist-test-slave-3kk6 krb5kdc[31718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:38:15.910775 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:15.898716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53612 (local address 127.16.235.126:33525)
0806 22:38:15.898932 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:15.898940 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:15.898971 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:38:15.899114 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:15.899122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:15.899178 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:15.899332 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:15.900108 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:15.901710 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:15.902860 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:15.903527 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:15.903634 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:15.906670 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:15.906674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:15.906675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:15.906689 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:15.908782 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:15.909189 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:15.909192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:15.909193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:15.909330 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:15.909557 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:15.909558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:15.909559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:15.909963 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:15.910142 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:15.910230 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:38:15.910516 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
I20260806 22:38:15.914878 31855 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:38:15.915633 31855 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:38:15.925434 31855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697: Generated new TSK 0
I20260806 22:38:15.925670 31855 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:38:15.965145 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33525
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:38:16.052613 31880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:16.052863 31880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:16.052891 31880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:16.054173 31880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:16.054220 31880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:16.054248 31880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:38:16.055841 31880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:16.056124 31880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:16.056381 31880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:16.061765 31890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:16.061759 31888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:16.061755 31886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:16.062057 31880 server_base.cc:1034] running on GCE node
I20260806 22:38:16.062283 31880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:16.062543 31880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:16.063712 31880 hybrid_clock.cc:648] HybridClock initialized: now 1786055896063717 us; error 52 us; skew 500 ppm
Aug 06 22:38:16 dist-test-slave-3kk6 krb5kdc[31718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055896, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:16.066294 31880 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:38:16.066828 31880 webserver.cc:460] Webserver started at http://127.16.235.65:46695/ using document root <none> and password file <none>
I20260806 22:38:16.067085 31880 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:16.067160 31880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:16.067328 31880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:16.068480 31880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "dc08782eb3194fdc898e4d698c47b3dc"
format_stamp: "Formatted at 2026-08-06 22:38:16 on dist-test-slave-3kk6"
server_key: "287e9560c60ace00d7006bab73680d64"
server_key_iv: "604b01f2b89f5483eca514da45bd5944"
server_key_version: "encryptionkey@0"
I20260806 22:38:16.068881 31880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "dc08782eb3194fdc898e4d698c47b3dc"
format_stamp: "Formatted at 2026-08-06 22:38:16 on dist-test-slave-3kk6"
server_key: "287e9560c60ace00d7006bab73680d64"
server_key_iv: "604b01f2b89f5483eca514da45bd5944"
server_key_version: "encryptionkey@0"
I20260806 22:38:16.070303 31880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:38:16.071229 31901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:16.071411 31880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:38:16.071523 31880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "dc08782eb3194fdc898e4d698c47b3dc"
format_stamp: "Formatted at 2026-08-06 22:38:16 on dist-test-slave-3kk6"
server_key: "287e9560c60ace00d7006bab73680d64"
server_key_iv: "604b01f2b89f5483eca514da45bd5944"
server_key_version: "encryptionkey@0"
I20260806 22:38:16.071601 31880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:16.082341 31880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:16.084522 31880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:16.084713 31880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:16.085008 31880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:16.085400 31880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:16.085461 31880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:16.085498 31880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:16.085541 31880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:16.091815 31880 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:44119
I20260806 22:38:16.091859 32068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:44119 every 8 connection(s)
I20260806 22:38:16.092250 31880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:38:16 dist-test-slave-3kk6 krb5kdc[31718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055896, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:38:16.101593 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 31880
I20260806 22:38:16.101696 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:38:16.102219 17325 external_mini_cluster.cc:1428] Setting key 0254bf4aec20e42afd2a41815942274e
I20260806 22:38:16.104975 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.093030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33525 (local address 127.16.235.65:58443)
0806 22:38:16.093232 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:16.093275 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:16.093783 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:16.094002 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:16.094005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:16.094606 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:16.094853 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.094858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.095637 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.095639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.097099 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.097103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.097508 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.097510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.097676 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.098423 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:16.098434 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:16.100610 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:16.103024 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:16.103033 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:16.103036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:16.103405 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:16.103650 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:16.103651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:16.103651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:16.103795 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:16.104155 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:16.104164 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:16.104343 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 22:38:16.104508 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 22:38:16.104708 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260806 22:38:16.104988 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.093112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:58443 (local address 127.16.235.126:33525)
0806 22:38:16.093202 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.093204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.093372 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:38:16.093868 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.093869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.093879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.093938 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:16.094950 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.095530 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.097189 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.097410 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.097484 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.100728 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:16.100730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:16.100730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:16.100739 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:16.102897 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:16.103496 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:16.103497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:16.103498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:16.103572 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:16.103860 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:16.103861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:16.103861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:16.104044 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:16.104121 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.104624 (+ 503us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.104797 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:38:16.105933 32071 heartbeater.cc:347] Connected to a master server at 127.16.235.126:33525
I20260806 22:38:16.106055 32071 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:16.106274 32071 heartbeater.cc:511] Master 127.16.235.126:33525 requested a full tablet report, sending...
I20260806 22:38:16.106916 31787 ts_manager.cc:194] Registered new tserver with Master: dc08782eb3194fdc898e4d698c47b3dc (127.16.235.65:44119)
I20260806 22:38:16.107539 31787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:58443
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:38:16.131397 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33525
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:38:16.218154 32079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:16.218405 32079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:16.218434 32079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:16.219722 32079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:16.219782 32079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:16.219813 32079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:38:16.221318 32079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:16.221531 32079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:16.221732 32079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:16.227115 32089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:16.227131 32087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:16.227126 32085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:16.227434 32079 server_base.cc:1034] running on GCE node
I20260806 22:38:16.227703 32079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:16.228026 32079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:16.229182 32079 hybrid_clock.cc:648] HybridClock initialized: now 1786055896229174 us; error 36 us; skew 500 ppm
Aug 06 22:38:16 dist-test-slave-3kk6 krb5kdc[31718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055896, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:16.232017 32079 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:38:16.232570 32079 webserver.cc:460] Webserver started at http://127.16.235.66:40923/ using document root <none> and password file <none>
I20260806 22:38:16.232815 32079 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:16.232887 32079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:16.233047 32079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:16.234192 32079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "584b87b6da6d4204a9302e71798fcaf2"
format_stamp: "Formatted at 2026-08-06 22:38:16 on dist-test-slave-3kk6"
server_key: "9df498154793bec87d74b1b0dea16de3"
server_key_iv: "f95d07ad28923c037fc5d25058472c67"
server_key_version: "encryptionkey@0"
I20260806 22:38:16.234608 32079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "584b87b6da6d4204a9302e71798fcaf2"
format_stamp: "Formatted at 2026-08-06 22:38:16 on dist-test-slave-3kk6"
server_key: "9df498154793bec87d74b1b0dea16de3"
server_key_iv: "f95d07ad28923c037fc5d25058472c67"
server_key_version: "encryptionkey@0"
I20260806 22:38:16.236117 32079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:38:16.237062 32098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:16.237272 32079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:38:16.237402 32079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "584b87b6da6d4204a9302e71798fcaf2"
format_stamp: "Formatted at 2026-08-06 22:38:16 on dist-test-slave-3kk6"
server_key: "9df498154793bec87d74b1b0dea16de3"
server_key_iv: "f95d07ad28923c037fc5d25058472c67"
server_key_version: "encryptionkey@0"
I20260806 22:38:16.237486 32079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:16.281498 32079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:16.283667 32079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:16.283859 32079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:16.284147 32079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:16.284538 32079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:16.284596 32079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:16.284641 32079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:16.284682 32079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:16.290939 32079 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:37805
I20260806 22:38:16.290987 32272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:37805 every 8 connection(s)
I20260806 22:38:16.291353 32079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:38:16 dist-test-slave-3kk6 krb5kdc[31718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055896, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:38:16.297978 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 32079
I20260806 22:38:16.298096 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:38:16.298349 17325 external_mini_cluster.cc:1428] Setting key b7deb23f6db994e2575e9b9af48b47c9
I20260806 22:38:16.301999 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.292229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:56059 (local address 127.16.235.126:33525)
0806 22:38:16.292307 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.292309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.292483 (+ 174us) server_negotiation.cc:407] Connection header received
0806 22:38:16.293005 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.293006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.293015 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.293081 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:16.294076 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.294614 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.295652 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.295846 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.295902 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.298366 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:16.298367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:16.298368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:16.298375 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:16.300137 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:16.300727 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:16.300727 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:16.300728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:16.300787 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:16.301112 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:16.301113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:16.301113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:16.301365 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:16.301440 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.301756 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.301870 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:38:16.302034 32275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.292167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33525 (local address 127.16.235.66:56059)
0806 22:38:16.292344 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:16.292389 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:16.292916 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:16.293168 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:16.293172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:16.293777 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:16.293994 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.294000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.294743 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.294744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.295570 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.295572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.295926 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.295927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.296030 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.296635 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:16.296644 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:16.298254 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:16.300252 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:16.300262 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:16.300265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:16.300632 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:16.300858 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:16.300859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:16.300860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:16.301031 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:16.301429 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:16.301436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:16.301603 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:38:16.301708 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:38:16.301851 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260806 22:38:16.302593 32273 heartbeater.cc:347] Connected to a master server at 127.16.235.126:33525
I20260806 22:38:16.302736 32273 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:16.302945 32273 heartbeater.cc:511] Master 127.16.235.126:33525 requested a full tablet report, sending...
I20260806 22:38:16.303421 31787 ts_manager.cc:194] Registered new tserver with Master: 584b87b6da6d4204a9302e71798fcaf2 (127.16.235.66:37805)
I20260806 22:38:16.304008 31787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:56059
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:38:16.327730 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33525
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:38:16.415578 32279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:38:16.415822 32279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:38:16.415848 32279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:38:16.417001 32279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:38:16.417044 32279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:38:16.417073 32279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:38:16.418511 32279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:38:16.418789 32279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:38:16.419008 32279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:38:16.424237 32288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:16.424300 32285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:38:16.424288 32286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:38:16.424459 32279 server_base.cc:1034] running on GCE node
I20260806 22:38:16.424672 32279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:38:16.424952 32279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:38:16.426174 32279 hybrid_clock.cc:648] HybridClock initialized: now 1786055896426155 us; error 41 us; skew 500 ppm
Aug 06 22:38:16 dist-test-slave-3kk6 krb5kdc[31718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055896, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:16.429103 32279 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:38:16.429544 32279 webserver.cc:460] Webserver started at http://127.16.235.67:44515/ using document root <none> and password file <none>
I20260806 22:38:16.429749 32279 fs_manager.cc:359] Metadata directory not provided
I20260806 22:38:16.429792 32279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:38:16.429896 32279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:38:16.431082 32279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "832895bdb974416da6376f3a4321a1ee"
format_stamp: "Formatted at 2026-08-06 22:38:16 on dist-test-slave-3kk6"
server_key: "a166ad0b7f5360552db88ade820c0ad5"
server_key_iv: "a22e6e0032e42c96ded01c58ed6b55da"
server_key_version: "encryptionkey@0"
I20260806 22:38:16.431432 32279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "832895bdb974416da6376f3a4321a1ee"
format_stamp: "Formatted at 2026-08-06 22:38:16 on dist-test-slave-3kk6"
server_key: "a166ad0b7f5360552db88ade820c0ad5"
server_key_iv: "a22e6e0032e42c96ded01c58ed6b55da"
server_key_version: "encryptionkey@0"
I20260806 22:38:16.432895 32279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:38:16.433717 32297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:16.433902 32279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:38:16.434007 32279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "832895bdb974416da6376f3a4321a1ee"
format_stamp: "Formatted at 2026-08-06 22:38:16 on dist-test-slave-3kk6"
server_key: "a166ad0b7f5360552db88ade820c0ad5"
server_key_iv: "a22e6e0032e42c96ded01c58ed6b55da"
server_key_version: "encryptionkey@0"
I20260806 22:38:16.434095 32279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:38:16.447579 32279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:38:16.450271 32279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:38:16.450446 32279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:38:16.450802 32279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:38:16.451179 32279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:38:16.451216 32279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:16.451260 32279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:38:16.451289 32279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:38:16.457705 32279 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:38641
I20260806 22:38:16.457752 32453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:38641 every 8 connection(s)
I20260806 22:38:16.458150 32279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:38:16.464382 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 32279
I20260806 22:38:16.464506 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:38:16.464704 17325 external_mini_cluster.cc:1428] Setting key 8b4c872155794a7f0792a0f4a82620ff
Aug 06 22:38:16 dist-test-slave-3kk6 krb5kdc[31718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055896, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:38:16.468611 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.459041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:47061 (local address 127.16.235.126:33525)
0806 22:38:16.459134 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.459137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.459338 (+ 201us) server_negotiation.cc:407] Connection header received
0806 22:38:16.459907 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.459909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.459920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.459986 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:16.461009 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.461528 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.462618 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.462938 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.463009 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.465730 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:16.465731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:16.465731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:16.465741 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:16.467291 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:16.467700 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:16.467701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:16.467701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:16.467761 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:16.467993 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:16.467994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:16.467994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:16.468107 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:16.468160 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.468449 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.468531 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:38:16.468694 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.458997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33525 (local address 127.16.235.67:47061)
0806 22:38:16.459193 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:16.459237 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:16.459780 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:16.460045 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:16.460049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:16.460669 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:38:16.460888 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.460893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.461637 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.461638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.462487 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.462490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.463025 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.463026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.463137 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.463681 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:38:16.463690 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:38:16.465631 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:38:16.467374 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:16.467387 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:16.467389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:16.467623 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:16.467826 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:38:16.467826 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:38:16.467826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:38:16.467934 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:38:16.468162 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:38:16.468166 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:38:16.468296 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 22:38:16.468383 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:38:16.468516 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260806 22:38:16.469262 32454 heartbeater.cc:347] Connected to a master server at 127.16.235.126:33525
I20260806 22:38:16.469373 32454 heartbeater.cc:464] Registering TS with master...
I20260806 22:38:16.469569 32454 heartbeater.cc:511] Master 127.16.235.126:33525 requested a full tablet report, sending...
I20260806 22:38:16.470146 31787 ts_manager.cc:194] Registered new tserver with Master: 832895bdb974416da6376f3a4321a1ee (127.16.235.67:38641)
I20260806 22:38:16.470525 31787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:47061
I20260806 22:38:16.476603 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:38:16.484638 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.477934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.16.235.126:33525)
0806 22:38:16.478027 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.478029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.478123 (+ 94us) server_negotiation.cc:407] Connection header received
0806 22:38:16.478212 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.478212 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.478225 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.478285 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:38:16.478934 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.479443 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.480171 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.480381 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.480471 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.481820 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:38:16.481822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:38:16.481822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:38:16.481829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:38:16.483381 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:16.483764 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:16.483765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:16.483766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:16.483830 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:38:16.484041 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:38:16.484042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:38:16.484042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:38:16.484214 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:38:16.484332 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.484435 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.484526 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:38:16.486485 31787 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:38:16.488147 31787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:38:16.495739 32476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.490591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:38641 (local address 127.0.0.1:36516)
0806 22:38:16.490973 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:16.490984 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:16.491045 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:16.491928 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:16.491929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:16.491931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:16.491961 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.491962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.492856 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.492858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.494292 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.494293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.495377 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.495378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.495473 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.495487 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:38:16.495571 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:38:16.495641 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":70,"thread_start_us":87,"threads_started":1}
I20260806 22:38:16.495780 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.490740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.16.235.67:38641)
0806 22:38:16.491690 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.491694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.491704 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:38:16.491743 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.491746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.491763 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.491839 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:16.492042 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.492735 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.494399 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.495191 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.495271 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.495334 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.495640 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.495689 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":887,"thread_start_us":100,"threads_started":1}
I20260806 22:38:16.496101 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.490830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:37805 (local address 127.0.0.1:35732)
0806 22:38:16.490973 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:16.490982 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:16.491022 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:16.491599 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:16.491600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:16.491603 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:16.491688 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.491689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.493067 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.493068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.494290 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.494293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.495377 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.495378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.495473 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.495487 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:38:16.495980 (+ 493us) client_negotiation.cc:240] Negotiation successful
0806 22:38:16.496028 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260806 22:38:16.496383 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.491068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35732 (local address 127.16.235.66:37805)
0806 22:38:16.491281 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.491286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.491297 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:38:16.491342 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.491343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.491366 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.491463 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:16.491789 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.492972 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.494398 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.495191 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.495271 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.495334 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.496237 (+ 903us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.496294 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
I20260806 22:38:16.496747 32361 tablet_service.cc:1467] Processing CreateTablet for tablet a744e0a9fb2c403d8c9ef8aefb34cafc (DEFAULT_TABLE table=test-table [id=e3a4aed2f46c4d04be993a339ed8c20d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:38:16.496997 32164 tablet_service.cc:1467] Processing CreateTablet for tablet a744e0a9fb2c403d8c9ef8aefb34cafc (DEFAULT_TABLE table=test-table [id=e3a4aed2f46c4d04be993a339ed8c20d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:38:16.497185 32361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a744e0a9fb2c403d8c9ef8aefb34cafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:16.497274 32164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a744e0a9fb2c403d8c9ef8aefb34cafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:16.499029 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.490591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:44119 (local address 127.0.0.1:38212)
0806 22:38:16.490790 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:16.490831 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:16.490927 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:16.491581 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:16.491585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:16.491588 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:16.491707 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.491710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.494422 (+ 2712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.494423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.495187 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.495189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.496167 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.496168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.498820 (+ 2652us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.498826 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:38:16.498896 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:38:16.498948 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":118,"threads_started":1}
I20260806 22:38:16.499313 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.490764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.16.235.65:44119)
0806 22:38:16.491346 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.491351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.491362 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:38:16.491407 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.491409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.491427 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.491508 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:16.491830 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.492952 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.495288 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.495986 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.496065 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.496127 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.499169 (+ 3042us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.499229 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":90,"threads_started":1}
I20260806 22:38:16.500057 31975 tablet_service.cc:1467] Processing CreateTablet for tablet a744e0a9fb2c403d8c9ef8aefb34cafc (DEFAULT_TABLE table=test-table [id=e3a4aed2f46c4d04be993a339ed8c20d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:38:16.500362 32478 tablet_bootstrap.cc:492] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: Bootstrap starting.
I20260806 22:38:16.500453 31975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a744e0a9fb2c403d8c9ef8aefb34cafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:38:16.501094 32478 tablet_bootstrap.cc:654] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:16.501539 32478 log.cc:824] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:16.501942 32479 tablet_bootstrap.cc:492] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: Bootstrap starting.
I20260806 22:38:16.502380 32478 tablet_bootstrap.cc:492] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: No bootstrap required, opened a new log
I20260806 22:38:16.502444 32478 ts_tablet_manager.cc:1397] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:38:16.502732 32479 tablet_bootstrap.cc:654] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:16.503245 32479 log.cc:824] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:16.504035 32482 tablet_bootstrap.cc:492] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc: Bootstrap starting.
I20260806 22:38:16.504202 32479 tablet_bootstrap.cc:492] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: No bootstrap required, opened a new log
I20260806 22:38:16.504278 32479 ts_tablet_manager.cc:1397] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:38:16.504570 32478 raft_consensus.cc:348] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } }
I20260806 22:38:16.504750 32478 raft_consensus.cc:374] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:16.504763 32482 tablet_bootstrap.cc:654] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc: Neither blocks nor log segments found. Creating new log.
I20260806 22:38:16.504796 32478 raft_consensus.cc:729] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 832895bdb974416da6376f3a4321a1ee, State: Initialized, Role: FOLLOWER
I20260806 22:38:16.504936 32478 consensus_queue.cc:260] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } }
I20260806 22:38:16.505252 32478 ts_tablet_manager.cc:1428] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:38:16.505263 32454 heartbeater.cc:503] Master 127.16.235.126:33525 was elected leader, sending a full tablet report...
I20260806 22:38:16.505312 32482 log.cc:824] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:38:16.506253 32482 tablet_bootstrap.cc:492] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc: No bootstrap required, opened a new log
I20260806 22:38:16.506349 32482 ts_tablet_manager.cc:1397] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:38:16.506702 32479 raft_consensus.cc:348] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } }
I20260806 22:38:16.506881 32479 raft_consensus.cc:374] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:16.506964 32479 raft_consensus.cc:729] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584b87b6da6d4204a9302e71798fcaf2, State: Initialized, Role: FOLLOWER
I20260806 22:38:16.507093 32479 consensus_queue.cc:260] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } }
I20260806 22:38:16.507392 32479 ts_tablet_manager.cc:1428] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:38:16.507493 32273 heartbeater.cc:503] Master 127.16.235.126:33525 was elected leader, sending a full tablet report...
I20260806 22:38:16.508510 32482 raft_consensus.cc:348] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } }
I20260806 22:38:16.508690 32482 raft_consensus.cc:374] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:38:16.508749 32482 raft_consensus.cc:729] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc08782eb3194fdc898e4d698c47b3dc, State: Initialized, Role: FOLLOWER
I20260806 22:38:16.508863 32482 consensus_queue.cc:260] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } }
I20260806 22:38:16.509161 32482 ts_tablet_manager.cc:1428] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:38:16.509249 32071 heartbeater.cc:503] Master 127.16.235.126:33525 was elected leader, sending a full tablet report...
W20260806 22:38:16.541953 32274 tablet.cc:2369] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:38:16.593055 32074 tablet.cc:2369] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:38:16.641455 32487 raft_consensus.cc:482] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:38:16.641646 32487 raft_consensus.cc:504] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } }
I20260806 22:38:16.642027 32487 leader_election.cc:288] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 832895bdb974416da6376f3a4321a1ee (127.16.235.67:38641), 584b87b6da6d4204a9302e71798fcaf2 (127.16.235.66:37805)
I20260806 22:38:16.645342 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.642406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:38133 (local address 127.16.235.66:37805)
0806 22:38:16.642453 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.642455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.642604 (+ 149us) server_negotiation.cc:407] Connection header received
0806 22:38:16.642680 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.642681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.642694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.642762 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:16.643006 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.643469 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.644533 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.644942 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.645030 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.645040 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.645217 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.645264 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:38:16.645344 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.642360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:54843 (local address 127.16.235.67:38641)
0806 22:38:16.642436 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.642438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.642576 (+ 138us) server_negotiation.cc:407] Connection header received
0806 22:38:16.642675 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.642676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.642686 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.642760 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:38:16.642971 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.643477 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.644556 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.644939 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.645031 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.645041 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.645217 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.645262 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:38:16.645390 32488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.642242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:38641 (local address 127.16.235.65:54843)
0806 22:38:16.642480 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:16.642495 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:16.642587 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:16.642780 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:16.642781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:16.642783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:16.642894 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.642896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.643587 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.643588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.644428 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.644431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.645047 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.645048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.645133 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.645139 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:38:16.645208 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:38:16.645265 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":52,"thread_start_us":121,"threads_started":1}
I20260806 22:38:16.645396 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.642262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:37805 (local address 127.16.235.65:38133)
0806 22:38:16.642496 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:16.642506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:38:16.642597 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:38:16.642781 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:38:16.642782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:38:16.642784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:38:16.642905 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.642907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.643556 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.643557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.644421 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:38:16.644424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.645043 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.645044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:38:16.645132 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.645136 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:38:16.645210 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:38:16.645263 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260806 22:38:16.645819 32389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a744e0a9fb2c403d8c9ef8aefb34cafc" candidate_uuid: "dc08782eb3194fdc898e4d698c47b3dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "832895bdb974416da6376f3a4321a1ee" is_pre_election: true
I20260806 22:38:16.645823 32197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a744e0a9fb2c403d8c9ef8aefb34cafc" candidate_uuid: "dc08782eb3194fdc898e4d698c47b3dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "584b87b6da6d4204a9302e71798fcaf2" is_pre_election: true
I20260806 22:38:16.646003 32389 raft_consensus.cc:2454] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc08782eb3194fdc898e4d698c47b3dc in term 0.
I20260806 22:38:16.646011 32197 raft_consensus.cc:2454] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc08782eb3194fdc898e4d698c47b3dc in term 0.
I20260806 22:38:16.646314 31906 leader_election.cc:302] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 832895bdb974416da6376f3a4321a1ee, dc08782eb3194fdc898e4d698c47b3dc; no voters:
I20260806 22:38:16.646515 32487 raft_consensus.cc:2781] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:38:16.646559 32487 raft_consensus.cc:482] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:38:16.646576 32487 raft_consensus.cc:3037] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:16.647492 32487 raft_consensus.cc:504] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } }
I20260806 22:38:16.647656 32487 leader_election.cc:288] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [CANDIDATE]: Term 1 election: Requested vote from peers 832895bdb974416da6376f3a4321a1ee (127.16.235.67:38641), 584b87b6da6d4204a9302e71798fcaf2 (127.16.235.66:37805)
I20260806 22:38:16.647806 32389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a744e0a9fb2c403d8c9ef8aefb34cafc" candidate_uuid: "dc08782eb3194fdc898e4d698c47b3dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "832895bdb974416da6376f3a4321a1ee"
I20260806 22:38:16.647883 32197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a744e0a9fb2c403d8c9ef8aefb34cafc" candidate_uuid: "dc08782eb3194fdc898e4d698c47b3dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "584b87b6da6d4204a9302e71798fcaf2"
I20260806 22:38:16.647909 32389 raft_consensus.cc:3037] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:16.647977 32197 raft_consensus.cc:3037] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:38:16.648737 32389 raft_consensus.cc:2454] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc08782eb3194fdc898e4d698c47b3dc in term 1.
I20260806 22:38:16.648787 32197 raft_consensus.cc:2454] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc08782eb3194fdc898e4d698c47b3dc in term 1.
I20260806 22:38:16.648977 31906 leader_election.cc:302] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 832895bdb974416da6376f3a4321a1ee, dc08782eb3194fdc898e4d698c47b3dc; no voters:
I20260806 22:38:16.649080 32487 raft_consensus.cc:2781] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:38:16.649266 32487 raft_consensus.cc:686] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 1 LEADER]: Becoming Leader. State: Replica: dc08782eb3194fdc898e4d698c47b3dc, State: Running, Role: LEADER
I20260806 22:38:16.649391 32487 consensus_queue.cc:237] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } }
I20260806 22:38:16.650383 31786 catalog_manager.cc:5495] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc reported cstate change: term changed from 0 to 1, leader changed from <none> to dc08782eb3194fdc898e4d698c47b3dc (127.16.235.65). New cstate: current_term: 1 leader_uuid: "dc08782eb3194fdc898e4d698c47b3dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc08782eb3194fdc898e4d698c47b3dc" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 44119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 } health_report { overall_health: UNKNOWN } } }
I20260806 22:38:16.677847 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.675171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.16.235.65:44119)
0806 22:38:16.675234 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:38:16.675236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:38:16.675247 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:38:16.675324 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:38:16.675325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:38:16.675346 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:38:16.675432 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:38:16.675598 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.676091 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.676926 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:38:16.677141 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:38:16.677271 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:38:16.677483 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:38:16.677600 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:38:16.677710 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:38:16.677730 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 22:38:16.677752 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:38:16.679854 32389 raft_consensus.cc:1261] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [term 1 FOLLOWER]: Refusing update from remote peer dc08782eb3194fdc898e4d698c47b3dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:38:16.679893 32197 raft_consensus.cc:1261] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [term 1 FOLLOWER]: Refusing update from remote peer dc08782eb3194fdc898e4d698c47b3dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:38:16.680339 32491 consensus_queue.cc:1035] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "832895bdb974416da6376f3a4321a1ee" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:38:16.680497 32487 consensus_queue.cc:1035] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "584b87b6da6d4204a9302e71798fcaf2" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 37805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:38:16.683468 32495 mvcc.cc:205] Tried to move back new op lower bound from 7315684952798007296 to 7315684952676626432. Current Snapshot: MvccSnapshot[applied={T|T < 7315684952798007296}]
I20260806 22:38:16.684230 32497 mvcc.cc:205] Tried to move back new op lower bound from 7315684952798007296 to 7315684952676626432. Current Snapshot: MvccSnapshot[applied={T|T < 7315684952798007296}]
I20260806 22:38:16.686133 31787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:38:16.686254 31787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:38:16.687156 31787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 974ee7e2815c48c8be751b08bbdca697: Sending DeleteTablet for 3 replicas of tablet a744e0a9fb2c403d8c9ef8aefb34cafc
I20260806 22:38:16.687546 32361 tablet_service.cc:1514] Processing DeleteTablet for tablet a744e0a9fb2c403d8c9ef8aefb34cafc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:38:16 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:36516
I20260806 22:38:16.687551 31975 tablet_service.cc:1514] Processing DeleteTablet for tablet a744e0a9fb2c403d8c9ef8aefb34cafc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:38:16 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:38212
I20260806 22:38:16.687798 32508 tablet_replica.cc:330] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc: stopping tablet replica
I20260806 22:38:16.687901 32508 raft_consensus.cc:2229] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:38:16.687877 32164 tablet_service.cc:1514] Processing DeleteTablet for tablet a744e0a9fb2c403d8c9ef8aefb34cafc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:38:16 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35732
I20260806 22:38:16.687956 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 31880
I20260806 22:38:16.688005 32508 raft_consensus.cc:2258] T a744e0a9fb2c403d8c9ef8aefb34cafc P dc08782eb3194fdc898e4d698c47b3dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:38:16.688169 32507 tablet_replica.cc:330] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: stopping tablet replica
I20260806 22:38:16.688273 32507 raft_consensus.cc:2229] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:38:16.688377 32507 raft_consensus.cc:2258] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:38:16.692332 32507 ts_tablet_manager.cc:1905] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:38:16.694272 32507 ts_tablet_manager.cc:1918] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:38:16.694378 32507 log.cc:1196] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/a744e0a9fb2c403d8c9ef8aefb34cafc
I20260806 22:38:16.694712 32507 ts_tablet_manager.cc:1939] T a744e0a9fb2c403d8c9ef8aefb34cafc P 832895bdb974416da6376f3a4321a1ee: Deleting consensus metadata
I20260806 22:38:16.695262 31766 catalog_manager.cc:4841] TS 832895bdb974416da6376f3a4321a1ee (127.16.235.67:38641): tablet a744e0a9fb2c403d8c9ef8aefb34cafc (table test-table [id=e3a4aed2f46c4d04be993a339ed8c20d]) successfully deleted
I20260806 22:38:16.703115 32509 tablet_replica.cc:330] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: stopping tablet replica
I20260806 22:38:16.703315 32509 raft_consensus.cc:2229] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:38:16.703490 32509 raft_consensus.cc:2258] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:38:16.704093 32509 ts_tablet_manager.cc:1905] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:38:16.706454 32509 ts_tablet_manager.cc:1918] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:38:16.706575 32509 log.cc:1196] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/a744e0a9fb2c403d8c9ef8aefb34cafc
I20260806 22:38:16.706948 32509 ts_tablet_manager.cc:1939] T a744e0a9fb2c403d8c9ef8aefb34cafc P 584b87b6da6d4204a9302e71798fcaf2: Deleting consensus metadata
I20260806 22:38:16.707825 31761 catalog_manager.cc:4841] TS 584b87b6da6d4204a9302e71798fcaf2 (127.16.235.66:37805): tablet a744e0a9fb2c403d8c9ef8aefb34cafc (table test-table [id=e3a4aed2f46c4d04be993a339ed8c20d]) successfully deleted
W20260806 22:38:16.709292 31761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:44119 (error 108)
I20260806 22:38:16.709514 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 32079
I20260806 22:38:16.712466 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:38:16.712262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:44119 (local address 127.0.0.1:38230)
0806 22:38:16.712344 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 22:38:16.712412 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:44119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 22:38:16.713269 31761 catalog_manager.cc:4584] TS dc08782eb3194fdc898e4d698c47b3dc (127.16.235.65:44119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a744e0a9fb2c403d8c9ef8aefb34cafc: Network error: Client connection negotiation failed: client connection to 127.16.235.65:44119: connect: Connection refused (error 111)
I20260806 22:38:16.719589 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 32279
I20260806 22:38:16.733929 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 31736
2026-08-06T22:38:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2348 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:38:16 dist-test-slave-3kk6 krb5kdc[32518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:38:16 dist-test-slave-3kk6 krb5kdc[32518](info): set up 2 sockets
Aug 06 22:38:16 dist-test-slave-3kk6 krb5kdc[32518](info): commencing operation
krb5kdc: starting...
W20260806 22:38:17.973709 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:38:18 dist-test-slave-3kk6 krb5kdc[32518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:38:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:38:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/rangeradmin_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/rangeradmin_127.16.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/rangerlookup_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/rangerlookup_127.16.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/HTTP_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/HTTP_127.16.235.84@KRBTEST.COM.keytab.
I20260806 22:38:18.165436 17325 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskVIGsRX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/postgres -l logfile start
2026-08-06 22:38:20.806 UTC [32565] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:38:20.807 UTC [32565] LOG: listening on IPv4 address "127.16.235.84", port 44223
2026-08-06 22:38:20.810 UTC [32565] LOG: listening on Unix socket "/tmp/.s.PGSQL.44223"
2026-08-06 22:38:20.847 UTC [32568] LOG: database system was shut down at 2026-08-06 22:38:19 UTC
2026-08-06 22:38:20.852 UTC [32565] LOG: database system is ready to accept connections
I20260806 22:38:21.971051 17325 mini_postgres.cc:96] Postgres bound to 44223
2026-08-06 22:38:21.979 UTC [32579] FATAL: database "slave" does not exist
127.16.235.84:44223 - accepting connections
I20260806 22:38:21.980479 17325 mini_ranger.cc:161] Starting Ranger...
I20260806 22:38:21.997220 17325 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:38:22.476609 17325 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:38:22.476687 17325 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-admin
2026-08-06 22:38:22,847 [I] DB FLAVOR :POSTGRES
2026-08-06 22:38:22,848 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:38:22,848 [I] Checking connection..
2026-08-06 22:38:22,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:38:23,104 [I] Checking connection passed.
2026-08-06 22:38:23,104 [I] --------- Verifying version history table ---------
2026-08-06 22:38:23,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:38:23,352 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:38:23,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:38:23,597 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:38:23,597 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:38:23,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:38:23.883 UTC [32681] WARNING: there is no transaction in progress
2026-08-06 22:38:23,891 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:38:23,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:38:24,162 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:38:24,163 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:38:24,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:38:24,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:38:24,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3kk6.c.gcp-upstream.internal','N') ;"
2026-08-06 22:38:24,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:38:25,225 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:38:25,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:38:25,495 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:38:25,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:38:25,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:38:26,028 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:38:26,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:38:27.001 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.018 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.035 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.051 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.068 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.087 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.145 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.152 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.166 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.177 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.187 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.202 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.212 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.219 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.226 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27.233 UTC [374] WARNING: there is no transaction in progress
2026-08-06 22:38:27,899 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:38:27,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:38:28,142 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:38:28,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:38:28,394 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:38:28,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:38:28,661 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:38:28,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:38:28,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:38:29,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3kk6.c.gcp-upstream.internal';"
2026-08-06 22:38:29,449 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:38:29,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:38:29,706 [I] DB_PATCHES have already been applied
I20260806 22:38:29.712644 17325 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-admin:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskVIGsRX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:38:29.712721 17325 mini_ranger.cc:193] Using host: 127.16.235.84
I20260806 22:38:29.716951 17325 mini_ranger.cc:227] Ranger admin URL: http://127.16.235.84:34559
Aug 06, 2026 10:38:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:38:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:38:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:38:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:38:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:38:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:38:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:38:31.083087 17325 mini_ranger.cc:160] Time spent starting Ranger: real 9.103s user 0.001s sys 0.011s
Aug 06, 2026 10:38:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:38:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 10:38:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:38:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:38:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:38:53.121018 17325 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/rangerkms_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/rangerkms_127.16.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.235.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.235.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.235.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/HTTP_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.235.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/HTTP_127.16.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:38:53 dist-test-slave-3kk6 krb5kdc[32518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055933, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:38:53.223676 17325 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:38:53.238480 17325 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:38:54.093401 17325 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:38:54.093499 17325 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-kms
I20260806 22:38:54.093523 17325 mini_ranger_kms.cc:227] Using postgres at 127.16.235.84:44223
2026-08-06 22:38:54,437 [I] DB FLAVOR :POSTGRES
2026-08-06 22:38:54,437 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:38:54,437 [I] Checking connection
2026-08-06 22:38:54,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:38:54,721 [I] connection success
2026-08-06 22:38:54,721 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:38:54,721 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:38:54,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:38:54,974 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:38:54,974 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:38:54,974 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:38:54,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:44223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:38:55,256 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:38:55.739539 17325 mini_ranger_kms.cc:310] Created kms service
I20260806 22:38:55.953667 17325 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:38:56.002825 17325 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:38:56 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055933, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:38:56.457998 17325 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:38:56.458141 17325 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-kms:/tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:38:56.458168 17325 mini_ranger_kms.cc:240] Using host: 127.16.235.84
I20260806 22:38:56.462756 17325 mini_ranger_kms.cc:276] Ranger KMS PID: 634
I20260806 22:38:56.462911 17325 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.235.84:37817
Aug 06, 2026 10:38:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:38:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:38:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:38:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:38:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:38:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 22:38:58 dist-test-slave-3kk6 krb5kdc[32518](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055938, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.235.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:38:59.228477 17325 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.005s user 0.006s sys 0.271s
I20260806 22:38:59.228622 17325 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:38:59.228710 17325 mini_ranger_kms.cc:401] 127.16.235.84:37817/kms/v1/keys
Aug 06 22:38:59 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055938, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.235.84@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
Aug 06 22:38:59 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055938, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.235.84@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
Aug 06 22:39:02 dist-test-slave-3kk6 krb5kdc[32518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:39:02.203327 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34413
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:34413
--ranger_config_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:37817/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:02.296783 708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:02.297010 708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:02.297034 708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:02.298137 708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:02.298177 708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:02.298192 708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:02.298204 708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:02.298218 708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:02.299979 708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.235.84:37817/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:34413
--ranger_config_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:02.300194 708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:02.300377 708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:02.306241 722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:02.306252 724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:02.306272 721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:02.306311 708 server_base.cc:1034] running on GCE node
I20260806 22:39:02.306979 708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:02.307310 708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:02.308497 708 hybrid_clock.cc:648] HybridClock initialized: now 1786055942308494 us; error 46 us; skew 500 ppm
Aug 06 22:39:02 dist-test-slave-3kk6 krb5kdc[32518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055942, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:02.315155 708 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:39:02.315662 708 webserver.cc:460] Webserver started at http://127.16.235.126:42659/ using document root <none> and password file <none>
I20260806 22:39:02.315938 708 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:02.316013 708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:02.316190 708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:39:02 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055942, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:02.366171 708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "53887fb95273400aa02edbe55873103c"
format_stamp: "Formatted at 2026-08-06 22:39:02 on dist-test-slave-3kk6"
server_key: "ac286561650cfeff4aeeb2548b564e11"
server_key_iv: "504b9994f833edd32c060349a81253f7"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:02.366712 708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "53887fb95273400aa02edbe55873103c"
format_stamp: "Formatted at 2026-08-06 22:39:02 on dist-test-slave-3kk6"
server_key: "ac286561650cfeff4aeeb2548b564e11"
server_key_iv: "504b9994f833edd32c060349a81253f7"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:02.368348 708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:02.381422 733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:02.381726 708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:02.381862 708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "53887fb95273400aa02edbe55873103c"
format_stamp: "Formatted at 2026-08-06 22:39:02 on dist-test-slave-3kk6"
server_key: "ac286561650cfeff4aeeb2548b564e11"
server_key_iv: "504b9994f833edd32c060349a81253f7"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:02.382017 708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:02.394131 708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:02.396484 708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:02.396660 708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:02.402478 708 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:34413
I20260806 22:39:02.402586 802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:34413 every 8 connection(s)
I20260806 22:39:02.403083 708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:02.403941 803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:02.406564 803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c: Bootstrap starting.
I20260806 22:39:02.407437 803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:02.407935 803 log.cc:824] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:02.408963 803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c: No bootstrap required, opened a new log
I20260806 22:39:02.411383 803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53887fb95273400aa02edbe55873103c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34413 } }
I20260806 22:39:02.411585 803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:02.411634 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 708
I20260806 22:39:02.411681 803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53887fb95273400aa02edbe55873103c, State: Initialized, Role: FOLLOWER
I20260806 22:39:02.411784 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:39:02.411890 803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53887fb95273400aa02edbe55873103c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34413 } }
I20260806 22:39:02.411960 803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:02.411984 803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:02.411978 17325 external_mini_cluster.cc:1428] Setting key 86024f4b4f26d4d560c4987ea17c643b
I20260806 22:39:02.412015 803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:02.412915 803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53887fb95273400aa02edbe55873103c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34413 } }
I20260806 22:39:02.413062 803 leader_election.cc:302] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53887fb95273400aa02edbe55873103c; no voters:
I20260806 22:39:02.413264 803 leader_election.cc:288] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:02.413511 810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:02.413614 803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:02.413710 810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [term 1 LEADER]: Becoming Leader. State: Replica: 53887fb95273400aa02edbe55873103c, State: Running, Role: LEADER
I20260806 22:39:02.413977 810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53887fb95273400aa02edbe55873103c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34413 } }
I20260806 22:39:02.416280 812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53887fb95273400aa02edbe55873103c. Latest consensus state: current_term: 1 leader_uuid: "53887fb95273400aa02edbe55873103c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53887fb95273400aa02edbe55873103c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34413 } } }
I20260806 22:39:02.416369 812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:02.416627 803 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:39:02.416741 831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:02.416626 811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53887fb95273400aa02edbe55873103c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53887fb95273400aa02edbe55873103c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34413 } } }
I20260806 22:39:02.417251 811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:02.417299 831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:39:02 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:02.430713 831 catalog_manager.cc:1347] Generated new cluster ID: 48f568c3a5f0469f88d5a7224a72a87e
I20260806 22:39:02.430863 831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:02.432471 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:02.412696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47960 (local address 127.16.235.126:34413)
0806 22:39:02.412908 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:02.412918 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:02.412958 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:39:02.413093 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:02.413101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:02.413164 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:02.413332 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:02.414503 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:02.416866 (+ 2363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:02.419430 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:02.421322 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:02.422051 (+ 729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:02.424913 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:02.424918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:02.424919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:02.424935 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:02.429974 (+ 5039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:02.430547 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:02.430551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:02.430551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:02.430722 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:02.431035 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:02.431036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:02.431036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:02.431367 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:02.431992 (+ 625us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:02.432064 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:39:02.432194 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":103,"threads_started":1}
I20260806 22:39:02.455097 831 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:02.455992 831 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:02.460870 831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c: Generated new TSK 0
I20260806 22:39:02.461093 831 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:39:03.238121 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:37817/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34413
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:03.371213 868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:03.371521 868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:03.371568 868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:03.373431 868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:03.373495 868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:03.373538 868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:39:03.375869 868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.235.84:37817/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:03.376159 868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:03.376506 868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:03.382472 877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:03.382529 878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:03.382440 880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:03.383085 868 server_base.cc:1034] running on GCE node
I20260806 22:39:03.383353 868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:03.383646 868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:03.384830 868 hybrid_clock.cc:648] HybridClock initialized: now 1786055943384813 us; error 39 us; skew 500 ppm
Aug 06 22:39:03 dist-test-slave-3kk6 krb5kdc[32518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:03.388092 868 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:39:03.388669 868 webserver.cc:460] Webserver started at http://127.16.235.65:37999/ using document root <none> and password file <none>
I20260806 22:39:03.388970 868 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:03.389060 868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:03.389230 868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:39:03 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:03.407261 868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "2dd28e3ce6d94295bf71294e3d40f056"
format_stamp: "Formatted at 2026-08-06 22:39:03 on dist-test-slave-3kk6"
server_key: "22e3396a47783b9f92e0e50fd3274577"
server_key_iv: "363e5acad8da1348b4006f67552cfee0"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:03.407917 868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "2dd28e3ce6d94295bf71294e3d40f056"
format_stamp: "Formatted at 2026-08-06 22:39:03 on dist-test-slave-3kk6"
server_key: "22e3396a47783b9f92e0e50fd3274577"
server_key_iv: "363e5acad8da1348b4006f67552cfee0"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:03.410048 868 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:39:03.424396 893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:03.424652 868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:39:03.424768 868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "2dd28e3ce6d94295bf71294e3d40f056"
format_stamp: "Formatted at 2026-08-06 22:39:03 on dist-test-slave-3kk6"
server_key: "22e3396a47783b9f92e0e50fd3274577"
server_key_iv: "363e5acad8da1348b4006f67552cfee0"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:03.424902 868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:03.460440 868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:03.463230 868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:03.463456 868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:03.463764 868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:03.464193 868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:03.464260 868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:03.464299 868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:03.464341 868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:03.472843 868 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:38129
I20260806 22:39:03.473265 868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:39:03.479607 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 868
I20260806 22:39:03.479744 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:39:03.479962 17325 external_mini_cluster.cc:1428] Setting key 08c913406d5211b5b8cacf25f90d6f5d
Aug 06 22:39:03 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:03.473791 1057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:38129 every 8 connection(s)
I20260806 22:39:03.484855 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:03.474449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:33561 (local address 127.16.235.126:34413)
0806 22:39:03.474585 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:03.474588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:03.474599 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:39:03.475230 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:03.475231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:03.475248 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:03.475343 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:03.475979 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:03.476537 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:03.477723 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:03.477945 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:03.478035 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:03.480518 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:03.480520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:03.480520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:03.480528 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:03.482514 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:03.483083 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:03.483084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:03.483085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:03.483154 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:03.483438 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:03.483439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:03.483440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:03.483582 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:03.483643 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:03.484647 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0806 22:39:03.484756 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260806 22:39:03.484395 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:03.474084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34413 (local address 127.16.235.65:33561)
0806 22:39:03.474278 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:03.474327 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:03.475043 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:03.475427 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:03.475432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:03.475649 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:03.475858 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:03.475863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:03.476650 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:03.476651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:03.477595 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:03.477598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:03.478043 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:03.478045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:03.478188 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:03.478851 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:03.478861 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:03.480402 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:03.482621 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:03.482628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:03.482631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:03.482986 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:03.483225 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:03.483226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:03.483226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:03.483363 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:03.483670 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:03.483676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:03.483862 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 22:39:03.483971 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:39:03.484135 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":118,"threads_started":1}
I20260806 22:39:03.486017 1058 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34413
I20260806 22:39:03.486138 1058 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:03.486370 1058 heartbeater.cc:511] Master 127.16.235.126:34413 requested a full tablet report, sending...
I20260806 22:39:03.487004 758 ts_manager.cc:194] Registered new tserver with Master: 2dd28e3ce6d94295bf71294e3d40f056 (127.16.235.65:38129)
I20260806 22:39:03.487974 758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:33561
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:39:03.520366 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:37817/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34413
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:39:03 dist-test-slave-3kk6 krb5kdc[32518](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:39:03.652221 1068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:03.652541 1068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:03.652580 1068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:03.654338 1068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:03.654450 1068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:03.654510 1068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:39:03.657083 1068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.235.84:37817/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:03.657368 1068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:03.657796 1068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:03.664889 1079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:03.665047 1082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:03.664932 1080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:03.666508 1068 server_base.cc:1034] running on GCE node
I20260806 22:39:03.666780 1068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:03.667083 1068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:03.668275 1068 hybrid_clock.cc:648] HybridClock initialized: now 1786055943668237 us; error 53 us; skew 500 ppm
Aug 06 22:39:03 dist-test-slave-3kk6 krb5kdc[32518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:03.672235 1068 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:39:03.672816 1068 webserver.cc:460] Webserver started at http://127.16.235.66:41941/ using document root <none> and password file <none>
I20260806 22:39:03.673108 1068 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:03.673195 1068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:03.673413 1068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:39:03 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:03.688553 1068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "29aea50473cc44f3aabe97b6395b9915"
format_stamp: "Formatted at 2026-08-06 22:39:03 on dist-test-slave-3kk6"
server_key: "2c234bba05b9d5e87fa556010cc3e517"
server_key_iv: "abc478315901f54ca67232791d6bc080"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:03.689080 1068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "29aea50473cc44f3aabe97b6395b9915"
format_stamp: "Formatted at 2026-08-06 22:39:03 on dist-test-slave-3kk6"
server_key: "2c234bba05b9d5e87fa556010cc3e517"
server_key_iv: "abc478315901f54ca67232791d6bc080"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:03.690970 1068 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:39:03.704445 1091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:03.704718 1068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:03.704829 1068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "29aea50473cc44f3aabe97b6395b9915"
format_stamp: "Formatted at 2026-08-06 22:39:03 on dist-test-slave-3kk6"
server_key: "2c234bba05b9d5e87fa556010cc3e517"
server_key_iv: "abc478315901f54ca67232791d6bc080"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:03.704941 1068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:03.751307 1068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:03.753839 1068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:03.754081 1068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:03.754372 1068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:03.754858 1068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:03.754964 1068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:03.755044 1068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:03.755105 1068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:03.764154 1068 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:33009
I20260806 22:39:03.764758 1068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:03.770030 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 1068
I20260806 22:39:03.770181 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:39:03.770402 17325 external_mini_cluster.cc:1428] Setting key 060961902f93ffc2558f7c2b26e9cf3d
Aug 06 22:39:03 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:03.765296 1271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:33009 every 8 connection(s)
I20260806 22:39:03.779117 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:03.766253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:40153 (local address 127.16.235.126:34413)
0806 22:39:03.766308 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:03.766310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:03.767125 (+ 815us) server_negotiation.cc:407] Connection header received
0806 22:39:03.767975 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:03.767977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:03.767994 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:03.768028 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:03.769021 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:03.769848 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:03.772106 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:03.772450 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:03.772499 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:03.775541 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:03.775543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:03.775544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:03.775552 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:03.777401 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:03.777931 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:03.777932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:03.777933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:03.777996 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:03.778301 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:03.778301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:03.778302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:03.778448 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:03.778502 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:03.778911 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 22:39:03.779014 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:39:03.779933 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:03.765930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34413 (local address 127.16.235.66:40153)
0806 22:39:03.766972 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:03.767020 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:03.767834 (+ 814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:03.768247 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:03.768251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:03.768597 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:03.768894 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:03.768901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:03.769989 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:03.769990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:03.771991 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:03.771996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:03.772601 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:03.772602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:03.772759 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:03.773417 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:03.773429 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:03.775410 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:03.777511 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:03.777519 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:03.777522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:03.777834 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:03.778066 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:03.778066 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:03.778067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:03.778217 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:03.778536 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:03.778542 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:03.778742 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 22:39:03.779505 (+ 763us) client_negotiation.cc:240] Negotiation successful
0806 22:39:03.779686 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"thread_start_us":89,"threads_started":1}
I20260806 22:39:03.780606 1272 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34413
I20260806 22:39:03.780707 1272 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:03.780907 1272 heartbeater.cc:511] Master 127.16.235.126:34413 requested a full tablet report, sending...
I20260806 22:39:03.781343 758 ts_manager.cc:194] Registered new tserver with Master: 29aea50473cc44f3aabe97b6395b9915 (127.16.235.66:33009)
I20260806 22:39:03.781841 758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:40153
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:39:03.823782 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:37817/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34413
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:03.951421 1278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:03.951830 1278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:03.951910 1278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:03.953802 1278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:03.953939 1278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:03.954027 1278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:39:03.956568 1278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.235.84:37817/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:03.957028 1278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:03.957403 1278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:03.964453 1288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:03.964613 1285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:03.964895 1286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:03.965315 1278 server_base.cc:1034] running on GCE node
I20260806 22:39:03.965560 1278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:03.965857 1278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:03.967061 1278 hybrid_clock.cc:648] HybridClock initialized: now 1786055943967021 us; error 39 us; skew 500 ppm
Aug 06 22:39:03 dist-test-slave-3kk6 krb5kdc[32518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:03.975071 1278 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:39:03.975692 1278 webserver.cc:460] Webserver started at http://127.16.235.67:44627/ using document root <none> and password file <none>
I20260806 22:39:03.976022 1278 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:03.976090 1278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:03.976351 1278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:39:03 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:03.993835 1278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2"
format_stamp: "Formatted at 2026-08-06 22:39:03 on dist-test-slave-3kk6"
server_key: "314c99f834657f54119ba99b82314bf4"
server_key_iv: "4cbeb45796e698df4a4da3d8fa31f09b"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:03.994482 1278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2"
format_stamp: "Formatted at 2026-08-06 22:39:03 on dist-test-slave-3kk6"
server_key: "314c99f834657f54119ba99b82314bf4"
server_key_iv: "4cbeb45796e698df4a4da3d8fa31f09b"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:03.999130 1278 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 22:39:04.028100 1302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:04.029223 1278 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260806 22:39:04.029430 1278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2"
format_stamp: "Formatted at 2026-08-06 22:39:03 on dist-test-slave-3kk6"
server_key: "314c99f834657f54119ba99b82314bf4"
server_key_iv: "4cbeb45796e698df4a4da3d8fa31f09b"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:04.029561 1278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:04.053303 1278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:04.056066 1278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:04.056303 1278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:04.056649 1278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:04.057081 1278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:04.057176 1278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:04.057251 1278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:04.057322 1278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:04.068979 1278 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:38655
I20260806 22:39:04.069594 1278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:39:04.070694 1468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:38655 every 8 connection(s)
I20260806 22:39:04.075896 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 1278
I20260806 22:39:04.076023 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:39:04.076241 17325 external_mini_cluster.cc:1428] Setting key 1b66b3d21e4f557e3bb183b1a81b61de
Aug 06 22:39:04 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:04.084098 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.072088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:42495 (local address 127.16.235.126:34413)
0806 22:39:04.072146 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:04.072149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:04.072552 (+ 403us) server_negotiation.cc:407] Connection header received
0806 22:39:04.073272 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:04.073274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:04.073288 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:04.073349 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:04.074460 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.075316 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.077324 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.077635 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.077679 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.080533 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:04.080535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:04.080535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:04.080543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:04.082366 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:04.082914 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:04.082915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:04.082915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:04.082977 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:04.083248 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:04.083249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:04.083249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:04.083397 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:04.083441 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:04.083893 (+ 452us) server_negotiation.cc:299] Negotiation successful
0806 22:39:04.083992 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:39:04.084951 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.072193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34413 (local address 127.16.235.67:42495)
0806 22:39:04.072395 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.072441 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:04.073141 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:04.073575 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:04.073579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:04.073928 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:04.074333 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.074340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.075442 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.075443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.077226 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.077229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.077776 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.077777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.077934 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.078616 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:04.078627 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:04.080408 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:04.082477 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:04.082485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:04.082488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:04.082821 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:04.083047 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:04.083048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:04.083049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:04.083176 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:04.083490 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:04.083497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:04.083732 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 22:39:04.084514 (+ 782us) client_negotiation.cc:240] Negotiation successful
0806 22:39:04.084696 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":125,"threads_started":1}
I20260806 22:39:04.085685 1469 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34413
I20260806 22:39:04.085789 1469 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:04.085995 1469 heartbeater.cc:511] Master 127.16.235.126:34413 requested a full tablet report, sending...
I20260806 22:39:04.086448 758 ts_manager.cc:194] Registered new tserver with Master: 5bca6ffc83964c8fbda9e5c7bf6e6dc2 (127.16.235.67:38655)
I20260806 22:39:04.087054 758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:42495
I20260806 22:39:04.088440 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:39:04.099707 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.090415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.16.235.126:34413)
0806 22:39:04.090458 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:04.090460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:04.090829 (+ 369us) server_negotiation.cc:407] Connection header received
0806 22:39:04.090927 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:04.090928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:04.090940 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:04.090973 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:04.092287 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.093026 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.094102 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.094426 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.094472 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.096411 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:04.096413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:04.096413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:04.096421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:04.098334 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:04.098829 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:04.098831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:04.098831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:04.098895 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:04.099093 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:04.099094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:04.099094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:04.099222 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:04.099305 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:04.099519 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 22:39:04.099603 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:39:04.102008 758 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:39:04.104532 758 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:39:04.159976 1485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.153942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:38129 (local address 127.0.0.1:36042)
0806 22:39:04.154498 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.154544 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:04.154609 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:04.155023 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:04.155026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:04.155030 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:04.155116 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.155118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.156794 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.156795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.158017 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.158020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.159646 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.159647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.159736 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.159751 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:39:04.159824 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:39:04.159883 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":22,"thread_start_us":93,"threads_started":1}
I20260806 22:39:04.161028 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.154705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36042 (local address 127.16.235.65:38129)
0806 22:39:04.154845 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:04.154850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:04.154861 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:39:04.154899 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:04.154901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:04.154919 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:04.155187 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:04.155241 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.156703 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.158108 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.159257 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.159343 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.159408 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:04.160896 (+ 1488us) server_negotiation.cc:299] Negotiation successful
0806 22:39:04.160946 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260806 22:39:04.162184 961 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac60c215ca84fd486bcf43b1d44ebcc (DEFAULT_TABLE table=test-table [id=7efead37edeb4549b9a5b2cc5feed269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:04.162585 961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac60c215ca84fd486bcf43b1d44ebcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:04.163216 1487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.153983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:33009 (local address 127.0.0.1:54594)
0806 22:39:04.155038 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.155050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:04.155116 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:04.155464 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:04.155466 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:04.155467 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:04.155511 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.155513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.156698 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.156700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.159180 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.159183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.162940 (+ 3757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.162941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.163037 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.163043 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:39:04.163098 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:39:04.163142 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"thread_start_us":71,"threads_started":1}
I20260806 22:39:04.163615 1488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.154486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.16.235.66:33009)
0806 22:39:04.155236 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:04.155241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:04.155252 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:39:04.155291 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:04.155293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:04.155308 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:04.155390 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:04.155581 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.156591 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.161916 (+ 5325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.162722 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.162812 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.162900 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:04.163463 (+ 563us) server_negotiation.cc:299] Negotiation successful
0806 22:39:04.163519 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":91,"threads_started":1}
I20260806 22:39:04.166230 1491 tablet_bootstrap.cc:492] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056: Bootstrap starting.
I20260806 22:39:04.167263 1491 tablet_bootstrap.cc:654] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:04.167421 1164 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac60c215ca84fd486bcf43b1d44ebcc (DEFAULT_TABLE table=test-table [id=7efead37edeb4549b9a5b2cc5feed269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:04.167673 1491 log.cc:824] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:04.167852 1164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac60c215ca84fd486bcf43b1d44ebcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:04.168514 1491 tablet_bootstrap.cc:492] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056: No bootstrap required, opened a new log
I20260806 22:39:04.168624 1491 ts_tablet_manager.cc:1397] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:39:04.171343 1491 raft_consensus.cc:348] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } }
I20260806 22:39:04.171607 1491 raft_consensus.cc:374] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:04.171711 1491 raft_consensus.cc:729] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dd28e3ce6d94295bf71294e3d40f056, State: Initialized, Role: FOLLOWER
I20260806 22:39:04.172026 1491 consensus_queue.cc:260] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } }
I20260806 22:39:04.172406 1491 ts_tablet_manager.cc:1428] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260806 22:39:04.172557 1058 heartbeater.cc:503] Master 127.16.235.126:34413 was elected leader, sending a full tablet report...
I20260806 22:39:04.173070 1494 tablet_bootstrap.cc:492] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915: Bootstrap starting.
I20260806 22:39:04.174242 1494 tablet_bootstrap.cc:654] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:04.174639 1494 log.cc:824] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:04.175501 1494 tablet_bootstrap.cc:492] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915: No bootstrap required, opened a new log
I20260806 22:39:04.175572 1494 ts_tablet_manager.cc:1397] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:39:04.177819 1494 raft_consensus.cc:348] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } }
I20260806 22:39:04.178026 1494 raft_consensus.cc:374] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:04.178061 1494 raft_consensus.cc:729] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29aea50473cc44f3aabe97b6395b9915, State: Initialized, Role: FOLLOWER
I20260806 22:39:04.178174 1494 consensus_queue.cc:260] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } }
I20260806 22:39:04.178432 1494 ts_tablet_manager.cc:1428] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:39:04.179570 1272 heartbeater.cc:503] Master 127.16.235.126:34413 was elected leader, sending a full tablet report...
I20260806 22:39:04.179903 1485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.154177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:38655 (local address 127.0.0.1:36868)
0806 22:39:04.160452 (+ 6275us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.160462 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:04.160501 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:04.160773 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:04.160774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:04.160776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:04.160807 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.160809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.177252 (+ 16443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.177254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.178424 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.178428 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.179666 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.179667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.179737 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.179742 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:39:04.179773 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 22:39:04.179822 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6264,"thread_start_us":57,"threads_started":1}
I20260806 22:39:04.180338 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.154854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36868 (local address 127.16.235.67:38655)
0806 22:39:04.159370 (+ 4516us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:04.159375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:04.160558 (+ 1183us) server_negotiation.cc:407] Connection header received
0806 22:39:04.160608 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:04.160610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:04.160627 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:04.160695 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:04.175892 (+ 15197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.177088 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.178566 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.179560 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.180078 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.180173 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:04.180201 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:39:04.180243 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4455,"thread_start_us":92,"threads_started":1}
I20260806 22:39:04.184767 1369 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac60c215ca84fd486bcf43b1d44ebcc (DEFAULT_TABLE table=test-table [id=7efead37edeb4549b9a5b2cc5feed269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:04.185197 1369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac60c215ca84fd486bcf43b1d44ebcc. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 22:39:04 dist-test-slave-3kk6 krb5kdc[32518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055943, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:04.192229 1498 tablet_bootstrap.cc:492] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: Bootstrap starting.
I20260806 22:39:04.193390 1498 tablet_bootstrap.cc:654] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:04.193743 1498 log.cc:824] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:04.199581 1498 tablet_bootstrap.cc:492] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: No bootstrap required, opened a new log
I20260806 22:39:04.199684 1498 ts_tablet_manager.cc:1397] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 22:39:04.201749 1498 raft_consensus.cc:348] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } }
I20260806 22:39:04.201961 1498 raft_consensus.cc:374] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:04.201997 1498 raft_consensus.cc:729] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bca6ffc83964c8fbda9e5c7bf6e6dc2, State: Initialized, Role: FOLLOWER
I20260806 22:39:04.202132 1498 consensus_queue.cc:260] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } }
I20260806 22:39:04.202446 1498 ts_tablet_manager.cc:1428] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:39:04.202574 1469 heartbeater.cc:503] Master 127.16.235.126:34413 was elected leader, sending a full tablet report...
W20260806 22:39:04.224539 1062 tablet.cc:2369] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:39:04.267225 1274 tablet.cc:2369] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:04.287209 1497 raft_consensus.cc:482] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:39:04.287384 1497 raft_consensus.cc:504] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } }
I20260806 22:39:04.287827 1497 leader_election.cc:288] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2dd28e3ce6d94295bf71294e3d40f056 (127.16.235.65:38129), 5bca6ffc83964c8fbda9e5c7bf6e6dc2 (127.16.235.67:38655)
I20260806 22:39:04.303658 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.288558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:53147 (local address 127.16.235.65:38129)
0806 22:39:04.288699 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:04.288702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:04.299838 (+ 11136us) server_negotiation.cc:407] Connection header received
0806 22:39:04.299964 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:04.299965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:04.299978 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:04.300014 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:04.300281 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.300965 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.302564 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.303214 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.303283 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.303293 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:04.303525 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 22:39:04.303571 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260806 22:39:04.304433 1502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.288033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:38129 (local address 127.16.235.66:53147)
0806 22:39:04.299678 (+ 11645us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.299700 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:04.299862 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:04.300095 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:04.300096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:04.300098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:04.300188 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.300190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.301106 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.301107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.302434 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.302438 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.303363 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.303364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.303440 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.303444 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:39:04.304284 (+ 840us) client_negotiation.cc:240] Negotiation successful
0806 22:39:04.304339 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11617,"thread_start_us":105,"threads_started":1}
I20260806 22:39:04.304966 986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac60c215ca84fd486bcf43b1d44ebcc" candidate_uuid: "29aea50473cc44f3aabe97b6395b9915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dd28e3ce6d94295bf71294e3d40f056" is_pre_election: true
I20260806 22:39:04.305258 986 raft_consensus.cc:2454] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29aea50473cc44f3aabe97b6395b9915 in term 0.
I20260806 22:39:04.305688 1093 leader_election.cc:302] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29aea50473cc44f3aabe97b6395b9915, 2dd28e3ce6d94295bf71294e3d40f056; no voters:
I20260806 22:39:04.305850 1497 raft_consensus.cc:2781] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:39:04.305977 1497 raft_consensus.cc:482] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:39:04.306013 1497 raft_consensus.cc:3037] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:04.307261 1497 raft_consensus.cc:504] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } }
I20260806 22:39:04.307410 1497 leader_election.cc:288] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [CANDIDATE]: Term 1 election: Requested vote from peers 2dd28e3ce6d94295bf71294e3d40f056 (127.16.235.65:38129), 5bca6ffc83964c8fbda9e5c7bf6e6dc2 (127.16.235.67:38655)
I20260806 22:39:04.307701 986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac60c215ca84fd486bcf43b1d44ebcc" candidate_uuid: "29aea50473cc44f3aabe97b6395b9915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dd28e3ce6d94295bf71294e3d40f056"
I20260806 22:39:04.307803 986 raft_consensus.cc:3037] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:04.308918 986 raft_consensus.cc:2454] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29aea50473cc44f3aabe97b6395b9915 in term 1.
I20260806 22:39:04.309218 1093 leader_election.cc:302] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29aea50473cc44f3aabe97b6395b9915, 2dd28e3ce6d94295bf71294e3d40f056; no voters:
I20260806 22:39:04.309352 1497 raft_consensus.cc:2781] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:04.309511 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.288650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:60033 (local address 127.16.235.67:38655)
0806 22:39:04.288773 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:04.288775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:04.299996 (+ 11221us) server_negotiation.cc:407] Connection header received
0806 22:39:04.300072 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:04.300074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:04.300086 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:04.300145 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:04.306395 (+ 6250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.307172 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.308563 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.309082 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.309136 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.309145 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:04.309396 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 22:39:04.309442 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 22:39:04.309549 1497 raft_consensus.cc:686] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 1 LEADER]: Becoming Leader. State: Replica: 29aea50473cc44f3aabe97b6395b9915, State: Running, Role: LEADER
I20260806 22:39:04.309656 1497 consensus_queue.cc:237] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } }
I20260806 22:39:04.310609 758 catalog_manager.cc:5495] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29aea50473cc44f3aabe97b6395b9915 (127.16.235.66). New cstate: current_term: 1 leader_uuid: "29aea50473cc44f3aabe97b6395b9915" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29aea50473cc44f3aabe97b6395b9915" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 33009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 } health_report { overall_health: UNKNOWN } } }
I20260806 22:39:04.312112 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.288271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:38655 (local address 127.16.235.66:60033)
0806 22:39:04.299914 (+ 11643us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.299925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:04.299990 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:04.306117 (+ 6127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:04.306119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:04.306122 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:04.306271 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.306273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.307353 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.307354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.308445 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:04.308448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.309210 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.309211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:04.309300 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.309304 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:39:04.309602 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 22:39:04.309647 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11628,"thread_start_us":40,"threads_started":1}
I20260806 22:39:04.312631 1397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac60c215ca84fd486bcf43b1d44ebcc" candidate_uuid: "29aea50473cc44f3aabe97b6395b9915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" is_pre_election: true
I20260806 22:39:04.312816 1397 raft_consensus.cc:2454] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29aea50473cc44f3aabe97b6395b9915 in term 0.
I20260806 22:39:04.312942 1398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac60c215ca84fd486bcf43b1d44ebcc" candidate_uuid: "29aea50473cc44f3aabe97b6395b9915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2"
I20260806 22:39:04.313017 1398 raft_consensus.cc:3037] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:04.314134 1398 raft_consensus.cc:2454] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29aea50473cc44f3aabe97b6395b9915 in term 1.
I20260806 22:39:04.319063 1488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.315518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54604 (local address 127.16.235.66:33009)
0806 22:39:04.315560 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:04.315562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:04.315599 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:39:04.315700 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:04.315701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:04.315722 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:04.315817 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:39:04.315970 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.316750 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.317898 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:04.318194 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:04.318264 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:04.318476 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:39:04.318608 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:39:04.318919 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:04.318939 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 22:39:04.318960 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:39:04.320328 1471 tablet.cc:2369] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:04.325430 1398 raft_consensus.cc:1261] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [term 1 FOLLOWER]: Refusing update from remote peer 29aea50473cc44f3aabe97b6395b9915: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:04.325898 1503 consensus_queue.cc:1035] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bca6ffc83964c8fbda9e5c7bf6e6dc2" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:04.327877 986 raft_consensus.cc:1261] T 0ac60c215ca84fd486bcf43b1d44ebcc P 2dd28e3ce6d94295bf71294e3d40f056 [term 1 FOLLOWER]: Refusing update from remote peer 29aea50473cc44f3aabe97b6395b9915: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:04.328243 1503 consensus_queue.cc:1035] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dd28e3ce6d94295bf71294e3d40f056" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:04.333945 1509 mvcc.cc:205] Tried to move back new op lower bound from 7315685147952791552 to 7315685147893121024. Current Snapshot: MvccSnapshot[applied={T|T < 7315685147952791552}]
I20260806 22:39:04.334141 1512 mvcc.cc:205] Tried to move back new op lower bound from 7315685147952791552 to 7315685147893121024. Current Snapshot: MvccSnapshot[applied={T|T < 7315685147952791552}]
I20260806 22:39:04.335299 1510 mvcc.cc:205] Tried to move back new op lower bound from 7315685147952791552 to 7315685147893121024. Current Snapshot: MvccSnapshot[applied={T|T < 7315685147952791552}]
I20260806 22:39:04.341440 758 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:04.341563 758 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:04.342417 758 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53887fb95273400aa02edbe55873103c: Sending DeleteTablet for 3 replicas of tablet 0ac60c215ca84fd486bcf43b1d44ebcc
I20260806 22:39:04.342806 961 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac60c215ca84fd486bcf43b1d44ebcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:04 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:36042
I20260806 22:39:04.343236 1164 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac60c215ca84fd486bcf43b1d44ebcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:04 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:54594
I20260806 22:39:04.343873 1369 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac60c215ca84fd486bcf43b1d44ebcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:04 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:36868
I20260806 22:39:04.344447 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 868
I20260806 22:39:04.354362 1521 tablet_replica.cc:330] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915: stopping tablet replica
I20260806 22:39:04.354547 1521 raft_consensus.cc:2229] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:39:04.354735 1521 raft_consensus.cc:2258] T 0ac60c215ca84fd486bcf43b1d44ebcc P 29aea50473cc44f3aabe97b6395b9915 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:04.355048 1522 tablet_replica.cc:330] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: stopping tablet replica
I20260806 22:39:04.355175 1522 raft_consensus.cc:2229] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:04.355297 1522 raft_consensus.cc:2258] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:04.355894 1522 ts_tablet_manager.cc:1905] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 22:39:04.366040 734 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:38129 (error 108)
I20260806 22:39:04.366832 1522 ts_tablet_manager.cc:1918] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:04.366945 1522 log.cc:1196] T 0ac60c215ca84fd486bcf43b1d44ebcc P 5bca6ffc83964c8fbda9e5c7bf6e6dc2: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/0ac60c215ca84fd486bcf43b1d44ebcc
I20260806 22:39:04.366935 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 1068
I20260806 22:39:04.404215 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 1278
I20260806 22:39:04.405335 1483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.405050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:33009 (local address 127.0.0.1:54612)
0806 22:39:04.405227 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.405290 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.66:33009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260806 22:39:04.405462 1483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:04.405167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:38129 (local address 127.0.0.1:36058)
0806 22:39:04.405424 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.405449 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:38129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244}
W20260806 22:39:04.405500 736 catalog_manager.cc:4584] TS 29aea50473cc44f3aabe97b6395b9915 (127.16.235.66:33009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ac60c215ca84fd486bcf43b1d44ebcc: Network error: Client connection negotiation failed: client connection to 127.16.235.66:33009: connect: Connection refused (error 111)
W20260806 22:39:04.416087 1485 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:39:04.415721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:38655 (local address 127.0.0.1:36874)
0806 22:39:04.415774 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:04.415787 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:04.415835 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:04.416011 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.67:38655: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:39:04.416685 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 708
I20260806 22:39:04.454711 17325 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:39:04.814558 17325 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:39:04.814786 17325 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:39:05.170305 17325 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:39:05.170 UTC [32565] LOG: received smart shutdown request
2026-08-06 22:39:05.175 UTC [32565] LOG: background worker "logical replication launcher" (PID 32576) exited with exit code 1
2026-08-06 22:39:05.175 UTC [32570] LOG: shutting down
2026-08-06 22:39:05.375 UTC [32565] LOG: database system is shut down
2026-08-06T22:39:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48753 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:05 dist-test-slave-3kk6 krb5kdc[1533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:05 dist-test-slave-3kk6 krb5kdc[1533](info): set up 2 sockets
Aug 06 22:39:05 dist-test-slave-3kk6 krb5kdc[1533](info): commencing operation
krb5kdc: starting...
W20260806 22:39:06.748587 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:39:06 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/rangeradmin_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/rangeradmin_127.16.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/rangerlookup_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/rangerlookup_127.16.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/HTTP_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/HTTP_127.16.235.84@KRBTEST.COM.keytab.
I20260806 22:39:06.938139 17325 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskVIGsRX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/postgres -l logfile start
2026-08-06 22:39:09.632 UTC [1578] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:39:09.632 UTC [1578] LOG: listening on IPv4 address "127.16.235.84", port 33661
2026-08-06 22:39:09.636 UTC [1578] LOG: listening on Unix socket "/tmp/.s.PGSQL.33661"
2026-08-06 22:39:09.670 UTC [1581] LOG: database system was shut down at 2026-08-06 22:39:08 UTC
2026-08-06 22:39:09.674 UTC [1578] LOG: database system is ready to accept connections
I20260806 22:39:10.801335 17325 mini_postgres.cc:96] Postgres bound to 33661
2026-08-06 22:39:10.809 UTC [1594] FATAL: database "slave" does not exist
127.16.235.84:33661 - accepting connections
I20260806 22:39:10.809857 17325 mini_ranger.cc:161] Starting Ranger...
I20260806 22:39:10.826965 17325 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:39:11.334095 17325 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:39:11.334194 17325 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-admin
2026-08-06 22:39:11,627 [I] DB FLAVOR :POSTGRES
2026-08-06 22:39:11,627 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:39:11,628 [I] Checking connection..
2026-08-06 22:39:11,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:39:11,874 [I] Checking connection passed.
2026-08-06 22:39:11,875 [I] --------- Verifying version history table ---------
2026-08-06 22:39:11,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:39:12,159 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:39:12,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:39:12,414 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:39:12,415 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:39:12,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:39:12.700 UTC [1693] WARNING: there is no transaction in progress
2026-08-06 22:39:12,709 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:39:12,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:39:12,977 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:39:12,977 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:39:12,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:39:13,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:39:13,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3kk6.c.gcp-upstream.internal','N') ;"
2026-08-06 22:39:13,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:39:14,042 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:39:14,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:39:14,336 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:39:14,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:39:14,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:39:14,877 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:39:14,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:39:15.898 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:15.915 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:15.932 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:15.948 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:15.968 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:15.989 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.058 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.065 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.081 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.092 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.105 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.120 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.130 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.137 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.145 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16.152 UTC [1844] WARNING: there is no transaction in progress
2026-08-06 22:39:16,848 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:39:16,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:39:17,094 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:39:17,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:39:17,376 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:39:17,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:39:17,635 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:39:17,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:39:17,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:39:18,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3kk6.c.gcp-upstream.internal';"
2026-08-06 22:39:18,421 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:39:18,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:39:18,694 [I] DB_PATCHES have already been applied
I20260806 22:39:18.700008 17325 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-admin:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskVIGsRX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:39:18.700081 17325 mini_ranger.cc:193] Using host: 127.16.235.84
I20260806 22:39:18.704141 17325 mini_ranger.cc:227] Ranger admin URL: http://127.16.235.84:50871
Aug 06, 2026 10:39:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:39:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:39:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:39:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:39:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:39:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:39:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:39:20.053370 17325 mini_ranger.cc:160] Time spent starting Ranger: real 9.244s user 0.000s sys 0.012s
Aug 06, 2026 10:39:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:39:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 10:39:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:39:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:39:41 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:39:42.260066 17325 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.235.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.235.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/rangerkms_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.235.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/rangerkms_127.16.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.235.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.235.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.235.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/HTTP_127.16.235.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.235.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/HTTP_127.16.235.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:39:42 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055982, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:39:42.355975 17325 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:39:42.367182 17325 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:39:43.285640 17325 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:39:43.285738 17325 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-kms
I20260806 22:39:43.285763 17325 mini_ranger_kms.cc:227] Using postgres at 127.16.235.84:33661
2026-08-06 22:39:43,468 [I] DB FLAVOR :POSTGRES
2026-08-06 22:39:43,469 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:39:43,469 [I] Checking connection
2026-08-06 22:39:43,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:39:43,727 [I] connection success
2026-08-06 22:39:43,728 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:39:43,728 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:39:43,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:39:43,987 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:39:43,988 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:39:43,988 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:39:43,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.235.84:33661/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:39:44,270 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:39:44.747099 17325 mini_ranger_kms.cc:310] Created kms service
I20260806 22:39:44.958920 17325 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:39:45.007637 17325 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:39:45 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055982, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:45.437942 17325 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:39:45.438190 17325 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-kms:/tmp/dist-test-taskVIGsRX/build/release/bin/postgresql.jar:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskVIGsRX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:39:45.438225 17325 mini_ranger_kms.cc:240] Using host: 127.16.235.84
I20260806 22:39:45.443181 17325 mini_ranger_kms.cc:276] Ranger KMS PID: 2098
I20260806 22:39:45.443274 17325 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.235.84:55433
Aug 06, 2026 10:39:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:39:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:39:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:39:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:39:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:39:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 22:39:47 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055987, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.235.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:48.135694 17325 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.780s user 0.000s sys 0.167s
I20260806 22:39:48.135828 17325 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:39:48.135885 17325 mini_ranger_kms.cc:401] 127.16.235.84:55433/kms/v1/keys
Aug 06 22:39:48 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055987, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.235.84@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
Aug 06 22:39:48 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055987, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.235.84@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
Aug 06 22:39:51 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:39:51.382124 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38477
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:38477
--ranger_config_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:55433/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:51.482503 2170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:51.482841 2170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:51.482872 2170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:51.484134 2170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:51.484189 2170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:51.484205 2170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:51.484216 2170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:51.484229 2170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:51.485962 2170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.235.84:55433/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:38477
--ranger_config_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:51.486246 2170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:51.486536 2170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:51.493332 2178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:51.493388 2179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:51.493511 2181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:51.493402 2170 server_base.cc:1034] running on GCE node
I20260806 22:39:51.493851 2170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:51.494233 2170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:51.495406 2170 hybrid_clock.cc:648] HybridClock initialized: now 1786055991495387 us; error 40 us; skew 500 ppm
Aug 06 22:39:51 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055991, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:51.501785 2170 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:39:51.502415 2170 webserver.cc:460] Webserver started at http://127.16.235.126:36897/ using document root <none> and password file <none>
I20260806 22:39:51.502715 2170 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:51.502782 2170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:51.502934 2170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:39:51 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055991, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:51.553429 2170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "52b939a2e69e4a9d967d2c0d5ecc521c"
format_stamp: "Formatted at 2026-08-06 22:39:51 on dist-test-slave-3kk6"
server_key: "3336bf79b21ad74817dbaa39460285e8"
server_key_iv: "850b268046d39792fc54e4228edbd937"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:51.553970 2170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "52b939a2e69e4a9d967d2c0d5ecc521c"
format_stamp: "Formatted at 2026-08-06 22:39:51 on dist-test-slave-3kk6"
server_key: "3336bf79b21ad74817dbaa39460285e8"
server_key_iv: "850b268046d39792fc54e4228edbd937"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:51.555939 2170 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:39:51.573499 2191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:51.573819 2170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:39:51.573943 2170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "52b939a2e69e4a9d967d2c0d5ecc521c"
format_stamp: "Formatted at 2026-08-06 22:39:51 on dist-test-slave-3kk6"
server_key: "3336bf79b21ad74817dbaa39460285e8"
server_key_iv: "850b268046d39792fc54e4228edbd937"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:51.574051 2170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:51.600795 2170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:51.603235 2170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:51.603430 2170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:51.609097 2170 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:38477
I20260806 22:39:51.609179 2277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:38477 every 8 connection(s)
I20260806 22:39:51.609643 2170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:51.610461 2278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:51.612140 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 2170
I20260806 22:39:51.612263 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:39:51.612471 17325 external_mini_cluster.cc:1428] Setting key 191c95539830fd623df180136c28afc2
I20260806 22:39:51.613692 2278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Bootstrap starting.
I20260806 22:39:51.614530 2278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:51.614923 2278 log.cc:824] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:51.615706 2278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: No bootstrap required, opened a new log
I20260806 22:39:51.618160 2278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } }
I20260806 22:39:51.618338 2278 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:51.618407 2278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b939a2e69e4a9d967d2c0d5ecc521c, State: Initialized, Role: FOLLOWER
I20260806 22:39:51.618577 2278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } }
I20260806 22:39:51.618739 2278 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:51.618818 2278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:51.618901 2278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:51.619613 2278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } }
I20260806 22:39:51.619750 2278 leader_election.cc:302] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52b939a2e69e4a9d967d2c0d5ecc521c; no voters:
I20260806 22:39:51.619994 2278 leader_election.cc:288] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:39:51.620028 2286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:51.620272 2286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 1 LEADER]: Becoming Leader. State: Replica: 52b939a2e69e4a9d967d2c0d5ecc521c, State: Running, Role: LEADER
I20260806 22:39:51.620404 2278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:51.620419 2286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } }
I20260806 22:39:51.620882 2286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52b939a2e69e4a9d967d2c0d5ecc521c. Latest consensus state: current_term: 1 leader_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } } }
Aug 06 22:39:51 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:51.621049 2286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:51.621224 2289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } } }
I20260806 22:39:51.621389 2289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:51.621558 2300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:51.622167 2300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:51.624598 2300 catalog_manager.cc:1347] Generated new cluster ID: 8d34ebe60830437aab8251aef722713e
I20260806 22:39:51.624711 2300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:51.625969 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:51.613131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.16.235.126:38477)
0806 22:39:51.613382 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:51.613393 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:51.613426 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:39:51.613978 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:51.613988 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:51.614060 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:51.614243 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:51.615139 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:51.616592 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:51.617765 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:51.618413 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:51.618913 (+ 500us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:51.621743 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:51.621748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:51.621748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:51.621763 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:51.623887 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:51.624330 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:51.624346 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:51.624346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:51.624446 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:51.624654 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:51.624655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:51.624655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:51.624953 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:51.625548 (+ 595us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:51.625624 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:39:51.625750 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":86,"threads_started":1}
I20260806 22:39:51.629987 2278 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:39:51.641793 2300 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:51.642841 2300 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:51.654779 2300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Generated new TSK 0
I20260806 22:39:51.655146 2300 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:39:52.407483 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:55433/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38477
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:52.515866 2345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:52.516100 2345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:52.516126 2345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:52.517416 2345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:52.517462 2345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:52.517491 2345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:39:52.519260 2345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.235.84:55433/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:52.519559 2345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:52.519829 2345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:52.525832 2354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:52.525832 2359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:52.526907 2345 server_base.cc:1034] running on GCE node
W20260806 22:39:52.527086 2352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:52.527371 2345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:52.527690 2345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:52.528894 2345 hybrid_clock.cc:648] HybridClock initialized: now 1786055992528882 us; error 34 us; skew 500 ppm
Aug 06 22:39:52 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:52.532323 2345 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:39:52.532932 2345 webserver.cc:460] Webserver started at http://127.16.235.65:37083/ using document root <none> and password file <none>
I20260806 22:39:52.533246 2345 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:52.533327 2345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:52.533507 2345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:39:52 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:52.550423 2345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "574dfa6705224e98ae24cdf61b18e1cf"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-3kk6"
server_key: "779ebde4c36c7464f7750104cd2ed57d"
server_key_iv: "6a7af17853da4325f37eb40e50a379fe"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:52.551196 2345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "574dfa6705224e98ae24cdf61b18e1cf"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-3kk6"
server_key: "779ebde4c36c7464f7750104cd2ed57d"
server_key_iv: "6a7af17853da4325f37eb40e50a379fe"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:52.553372 2345 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:39:52.568511 2366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:52.568778 2345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:39:52.568904 2345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "574dfa6705224e98ae24cdf61b18e1cf"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-3kk6"
server_key: "779ebde4c36c7464f7750104cd2ed57d"
server_key_iv: "6a7af17853da4325f37eb40e50a379fe"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:52.569039 2345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:52.640672 2345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:52.643208 2345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:52.643396 2345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:52.643708 2345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:52.644115 2345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:52.644164 2345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:52.644207 2345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:52.644232 2345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:52.651981 2345 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:33675
I20260806 22:39:52.652400 2345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:39:52 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:52.660748 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 2345
I20260806 22:39:52.660878 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:39:52.661111 17325 external_mini_cluster.cc:1428] Setting key 5db497cee9465e4edd5f2b2ee704ff57
I20260806 22:39:52.664286 2529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:52.653648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:43837 (local address 127.16.235.126:38477)
0806 22:39:52.654544 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:52.654546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:52.654554 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:39:52.654577 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:52.654577 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:52.654590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:52.654748 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:52.655401 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.655931 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.656990 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.657197 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.657284 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:52.659648 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:52.659650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:52.659650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:52.659659 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:52.661889 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:52.662456 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:52.662457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:52.662458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:52.662525 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:52.662869 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:52.662870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:52.662870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:52.663031 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:52.663094 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:52.664049 (+ 955us) server_negotiation.cc:299] Negotiation successful
0806 22:39:52.664162 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":112,"threads_started":1}
I20260806 22:39:52.653445 2522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:33675 every 8 connection(s)
I20260806 22:39:52.663941 2528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:52.653280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:38477 (local address 127.16.235.65:43837)
0806 22:39:52.653808 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:52.653866 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:52.654377 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:52.654849 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:52.654856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:52.655078 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:52.655285 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:52.655290 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.656046 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.656048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:52.656889 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:52.656891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.657324 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.657326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:52.657458 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:52.658063 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:52.658074 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:52.659538 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:52.661985 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:52.661995 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:52.661999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:52.662365 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:52.662591 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:52.662592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:52.662592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:52.662790 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:52.663121 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:52.663129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:52.663313 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 22:39:52.663424 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:39:52.663611 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":111,"threads_started":1}
I20260806 22:39:52.672583 2524 heartbeater.cc:347] Connected to a master server at 127.16.235.126:38477
I20260806 22:39:52.672708 2524 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:52.672943 2524 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
I20260806 22:39:52.673662 2219 ts_manager.cc:194] Registered new tserver with Master: 574dfa6705224e98ae24cdf61b18e1cf (127.16.235.65:33675)
I20260806 22:39:52.674625 2219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:43837
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:39:52.706429 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:55433/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38477
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:39:52 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:39:52.831980 2537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:52.832305 2537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:52.832347 2537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:52.834190 2537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:52.834262 2537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:52.834306 2537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:39:52.836702 2537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.235.84:55433/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:52.836994 2537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:52.837304 2537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:52.845250 2553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:52.845439 2551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:52.845621 2550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:52.846236 2537 server_base.cc:1034] running on GCE node
I20260806 22:39:52.846524 2537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:52.846858 2537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:52.848047 2537 hybrid_clock.cc:648] HybridClock initialized: now 1786055992848004 us; error 59 us; skew 500 ppm
Aug 06 22:39:52 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:52.851639 2537 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:39:52.852224 2537 webserver.cc:460] Webserver started at http://127.16.235.66:45483/ using document root <none> and password file <none>
I20260806 22:39:52.852541 2537 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:52.852597 2537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:52.852736 2537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:39:52 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:52.869890 2537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "db3ca75bb94d41eea389485f3bcbb19d"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-3kk6"
server_key: "b12fd1190efe557543e0b7d33a021c1c"
server_key_iv: "d817c900192250473c33444d7a790473"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:52.870415 2537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "db3ca75bb94d41eea389485f3bcbb19d"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-3kk6"
server_key: "b12fd1190efe557543e0b7d33a021c1c"
server_key_iv: "d817c900192250473c33444d7a790473"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:52.879217 2537 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.001s
I20260806 22:39:52.901043 2563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:52.901309 2537 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.000s
I20260806 22:39:52.901435 2537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "db3ca75bb94d41eea389485f3bcbb19d"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-3kk6"
server_key: "b12fd1190efe557543e0b7d33a021c1c"
server_key_iv: "d817c900192250473c33444d7a790473"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:52.901561 2537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:52.913065 2537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:52.915844 2537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:52.916018 2537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:52.916319 2537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:52.916749 2537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:52.916805 2537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:52.916841 2537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:52.916860 2537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:52.925927 2537 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:40111
I20260806 22:39:52.926445 2537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:52.931520 2724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:40111 every 8 connection(s)
I20260806 22:39:52.936635 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 2537
I20260806 22:39:52.936779 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:39:52.936980 17325 external_mini_cluster.cc:1428] Setting key 9b05fb3324d47f5f69ca9df910283636
Aug 06 22:39:52 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:52.944729 2529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:52.932411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:39005 (local address 127.16.235.126:38477)
0806 22:39:52.932481 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:52.932483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:52.932810 (+ 327us) server_negotiation.cc:407] Connection header received
0806 22:39:52.933583 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:52.933585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:52.933604 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:52.933639 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:52.934630 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.935442 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.937031 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.937353 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.937407 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:52.940755 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:52.940758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:52.940758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:52.940768 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:52.942729 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:52.943358 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:52.943359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:52.943360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:52.943431 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:52.943773 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:52.943774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:52.943774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:52.943958 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:52.944008 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:52.944501 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 22:39:52.944617 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:39:52.945398 2729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:52.932377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:38477 (local address 127.16.235.66:39005)
0806 22:39:52.932618 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:52.932677 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:52.933432 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:52.933861 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:52.933866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:52.934174 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:52.934496 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:52.934504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.935579 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.935580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:52.936914 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:52.936918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:52.937523 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:52.937524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:52.937672 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:52.938766 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:52.938778 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:52.940605 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:52.942848 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:52.942859 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:52.942863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:52.943233 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:52.943514 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:52.943515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:52.943516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:52.943681 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:52.944100 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:52.944109 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:52.944318 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 22:39:52.944892 (+ 574us) client_negotiation.cc:240] Negotiation successful
0806 22:39:52.945127 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":119,"threads_started":1}
I20260806 22:39:52.947799 2725 heartbeater.cc:347] Connected to a master server at 127.16.235.126:38477
I20260806 22:39:52.947938 2725 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:52.948167 2725 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
I20260806 22:39:52.948570 2219 ts_manager.cc:194] Registered new tserver with Master: db3ca75bb94d41eea389485f3bcbb19d (127.16.235.66:40111)
I20260806 22:39:52.949156 2219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:39005
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:39:52.981858 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:55433/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38477
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:53.099454 2733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:53.099797 2733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:53.099839 2733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:53.101706 2733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:53.101811 2733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:53.101858 2733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:39:53.104403 2733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.235.84:55433/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:53.104727 2733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:53.105064 2733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:53.111977 2743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:53.112331 2740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:53.112548 2739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:53.112907 2733 server_base.cc:1034] running on GCE node
I20260806 22:39:53.113158 2733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:53.113503 2733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:53.114718 2733 hybrid_clock.cc:648] HybridClock initialized: now 1786055993114689 us; error 55 us; skew 500 ppm
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:53.118562 2733 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:39:53.119413 2733 webserver.cc:460] Webserver started at http://127.16.235.67:34447/ using document root <none> and password file <none>
I20260806 22:39:53.119803 2733 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:53.119942 2733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:53.120159 2733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:53.135732 2733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "9713f837fbed4741a711b7e1e049ae81"
format_stamp: "Formatted at 2026-08-06 22:39:53 on dist-test-slave-3kk6"
server_key: "2ba707c4edfee315b1c5d893775b0530"
server_key_iv: "2b83fff7897e6ae9f51304118eac98d4"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:53.136268 2733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "9713f837fbed4741a711b7e1e049ae81"
format_stamp: "Formatted at 2026-08-06 22:39:53 on dist-test-slave-3kk6"
server_key: "2ba707c4edfee315b1c5d893775b0530"
server_key_iv: "2b83fff7897e6ae9f51304118eac98d4"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:53.147324 2733 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.003s sys 0.001s
I20260806 22:39:53.170483 2752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.170805 2733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:39:53.170908 2733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "9713f837fbed4741a711b7e1e049ae81"
format_stamp: "Formatted at 2026-08-06 22:39:53 on dist-test-slave-3kk6"
server_key: "2ba707c4edfee315b1c5d893775b0530"
server_key_iv: "2b83fff7897e6ae9f51304118eac98d4"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:53.171011 2733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:53.192608 2733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:53.195031 2733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:53.195219 2733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:53.195518 2733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:53.196003 2733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:53.196060 2733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.196102 2733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:53.196130 2733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.204077 2733 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:45227
I20260806 22:39:53.204566 2733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:39:53.205921 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 2733
I20260806 22:39:53.206044 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:39:53.206259 17325 external_mini_cluster.cc:1428] Setting key 018d2deec7d4c93f9beff2b95d712f1a
I20260806 22:39:53.207831 2897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:45227 every 8 connection(s)
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:53.222288 2529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:53.208740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:53555 (local address 127.16.235.126:38477)
0806 22:39:53.208794 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:53.208796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:53.211003 (+ 2207us) server_negotiation.cc:407] Connection header received
0806 22:39:53.211868 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:53.211870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:53.211885 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:53.211920 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:53.212711 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.213437 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.214970 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.215260 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.215314 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:53.218491 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:53.218493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:53.218494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:53.218503 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:53.220424 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:53.220990 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:53.220991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:53.220992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:53.221058 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:53.221399 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:53.221400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:53.221400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:53.221566 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:53.221612 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:53.222055 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 22:39:53.222156 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:39:53.223657 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:53.208768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:38477 (local address 127.16.235.67:53555)
0806 22:39:53.210825 (+ 2057us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:53.210876 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:53.211687 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:53.212110 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:53.212114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:53.212384 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:53.212596 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:53.212601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.213568 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.213569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:53.214850 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:53.214854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.215499 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.215500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:53.215660 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:53.216395 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:53.216406 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:53.218362 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:53.220558 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:53.220566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:53.220569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:53.220891 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:53.221163 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:53.221164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:53.221164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:53.221322 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:53.221691 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:53.221698 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:53.221883 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 22:39:53.222932 (+ 1049us) client_negotiation.cc:240] Negotiation successful
0806 22:39:53.223339 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1998,"thread_start_us":89,"threads_started":1}
I20260806 22:39:53.224475 2898 heartbeater.cc:347] Connected to a master server at 127.16.235.126:38477
I20260806 22:39:53.224613 2898 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:53.224824 2898 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
I20260806 22:39:53.225483 2217 ts_manager.cc:194] Registered new tserver with Master: 9713f837fbed4741a711b7e1e049ae81 (127.16.235.67:45227)
I20260806 22:39:53.226015 2217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:53555
I20260806 22:39:53.236384 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:39:53.237265 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 2345
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:53.267407 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 2537
I20260806 22:39:53.280187 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 2733
I20260806 22:39:53.288753 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 2170
I20260806 22:39:53.306910 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38477
--webserver_interface=127.16.235.126
--webserver_port=36897
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:38477
--ranger_config_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:55433/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.235.84:55433/kms with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:53.407779 2904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:53.408022 2904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:53.408046 2904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:53.409176 2904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:53.409219 2904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:53.409233 2904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:53.409246 2904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:53.409258 2904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:53.411000 2904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.235.84:55433/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:38477
--ranger_config_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:53.411217 2904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:53.411406 2904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:53.416550 2917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:53.416559 2916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:53.416540 2921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:53.416775 2904 server_base.cc:1034] running on GCE node
I20260806 22:39:53.417102 2904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:53.417459 2904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:53.418637 2904 hybrid_clock.cc:648] HybridClock initialized: now 1786055993418615 us; error 41 us; skew 500 ppm
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:53.421437 2904 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:39:53.421995 2904 webserver.cc:460] Webserver started at http://127.16.235.126:36897/ using document root <none> and password file <none>
I20260806 22:39:53.422261 2904 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:53.422346 2904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:53.465091 2904 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:53.466001 2930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.466212 2904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:53.466325 2904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "52b939a2e69e4a9d967d2c0d5ecc521c"
format_stamp: "Formatted at 2026-08-06 22:39:51 on dist-test-slave-3kk6"
server_key: "3336bf79b21ad74817dbaa39460285e8"
server_key_iv: "850b268046d39792fc54e4228edbd937"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:53.466852 2904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:53.482839 2904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:53.485401 2904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:53.485612 2904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:53.490578 2904 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:38477
I20260806 22:39:53.490618 3010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:38477 every 8 connection(s)
I20260806 22:39:53.491104 2904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:53.492846 3011 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:39:53.493716 3011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Bootstrap starting.
I20260806 22:39:53.495070 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 2904
I20260806 22:39:53.495398 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:55433/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:33675
--local_ip_for_outbound_sockets=127.16.235.65
--tserver_master_addrs=127.16.235.126:38477
--webserver_port=37083
--webserver_interface=127.16.235.65
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.235.84:55433/kms with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:39:53.504592 3011 log.cc:824] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:53.507323 3011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:39:53.507699 3011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Bootstrap complete.
I20260806 22:39:53.511396 3011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } }
I20260806 22:39:53.511760 3011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b939a2e69e4a9d967d2c0d5ecc521c, State: Initialized, Role: FOLLOWER
I20260806 22:39:53.511943 3011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } }
I20260806 22:39:53.512017 3011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:39:53.512058 3011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:39:53.512104 3011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:39:53.513545 3011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } }
I20260806 22:39:53.513672 3011 leader_election.cc:302] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52b939a2e69e4a9d967d2c0d5ecc521c; no voters:
I20260806 22:39:53.513866 3011 leader_election.cc:288] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:39:53.513964 3017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:39:53.514142 3011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:53.514241 3017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [term 2 LEADER]: Becoming Leader. State: Replica: 52b939a2e69e4a9d967d2c0d5ecc521c, State: Running, Role: LEADER
I20260806 22:39:53.514393 3017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } }
I20260806 22:39:53.514837 3019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } } }
I20260806 22:39:53.514927 3019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:53.515139 3022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52b939a2e69e4a9d967d2c0d5ecc521c. Latest consensus state: current_term: 2 leader_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b939a2e69e4a9d967d2c0d5ecc521c" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38477 } } }
I20260806 22:39:53.515204 3022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:53.515360 3036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:53.515753 3011 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:39:53.515944 3036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:53.516142 3036 catalog_manager.cc:1259] Loaded cluster ID: 8d34ebe60830437aab8251aef722713e
I20260806 22:39:53.516252 3036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:53.521162 3036 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:53.521606 3036 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Loaded TSK: 0
I20260806 22:39:53.521960 3036 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:39:53.622104 3014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:53.622457 3014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:53.622501 3014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:53.624526 3014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:53.624609 3014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:53.624660 3014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:39:53.627297 3014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.235.84:55433/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:33675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=37083
--tserver_master_addrs=127.16.235.126:38477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:53.627765 3014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:53.628134 3014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:53.635643 3069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:53.636546 3068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:53.637118 3072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:53.637410 3014 server_base.cc:1034] running on GCE node
I20260806 22:39:53.637658 3014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:53.637965 3014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:53.639135 3014 hybrid_clock.cc:648] HybridClock initialized: now 1786055993639114 us; error 39 us; skew 500 ppm
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:53.642885 3014 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:39:53.643539 3014 webserver.cc:460] Webserver started at http://127.16.235.65:37083/ using document root <none> and password file <none>
I20260806 22:39:53.643895 3014 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:53.644003 3014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:53.662336 3014 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:53.663857 3087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.664134 3014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:39:53.664270 3014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "574dfa6705224e98ae24cdf61b18e1cf"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-3kk6"
server_key: "779ebde4c36c7464f7750104cd2ed57d"
server_key_iv: "6a7af17853da4325f37eb40e50a379fe"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:53.664832 3014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:53.708659 3014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:53.711716 3014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:53.711997 3014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:53.712361 3014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:53.712842 3014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:53.712957 3014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.713086 3014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:53.713194 3014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.721163 3014 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:33675
I20260806 22:39:53.721855 3014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:39:53.724435 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 3014
I20260806 22:39:53.724789 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:55433/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:40111
--local_ip_for_outbound_sockets=127.16.235.66
--tserver_master_addrs=127.16.235.126:38477
--webserver_port=45483
--webserver_interface=127.16.235.66
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.235.84:55433/kms with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:39:53.727520 3254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:33675 every 8 connection(s)
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:53.748204 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:53.728301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:38477 (local address 127.16.235.65:56497)
0806 22:39:53.732475 (+ 4174us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:53.732542 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:53.733248 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:53.734164 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:53.734169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:53.734473 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:53.734840 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:53.734848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.736708 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.736709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:53.738355 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:53.738360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.739543 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.739545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:53.739721 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:53.740486 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:53.740499 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:53.742800 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:53.745707 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:53.745717 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:53.745719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:53.746089 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:53.746502 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:53.746503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:53.746504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:53.746722 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:53.747411 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:53.747419 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:53.747602 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 22:39:53.747716 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:39:53.747913 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4112,"thread_start_us":118,"threads_started":1}
I20260806 22:39:53.749342 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:53.728740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:56497 (local address 127.16.235.126:38477)
0806 22:39:53.733509 (+ 4769us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:53.733520 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:53.733566 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:39:53.733740 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:53.733749 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:53.733814 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:53.733985 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:53.735004 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.736549 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.738523 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.739374 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.740109 (+ 735us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:53.742973 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:53.742979 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:53.742980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:53.742997 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:53.745566 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:53.746235 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:53.746241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:53.746242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:53.746393 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:53.746826 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:53.746828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:53.746829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:53.747262 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:53.747371 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:53.748944 (+ 1573us) server_negotiation.cc:299] Negotiation successful
0806 22:39:53.749102 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4699,"thread_start_us":90,"threads_started":1}
I20260806 22:39:53.750353 3256 heartbeater.cc:347] Connected to a master server at 127.16.235.126:38477
I20260806 22:39:53.750461 3256 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:53.750788 3256 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
W20260806 22:39:53.751804 3256 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:38477 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:39:53.751883 3256 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:53.751953 3256 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
I20260806 22:39:53.752400 3256 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:53.752491 3256 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
W20260806 22:39:53.752909 3256 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:39:53.856608 3260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:53.856977 3260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:53.857023 3260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:53.859189 3260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:53.859274 3260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:53.859342 3260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:39:53.862063 3260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.235.84:55433/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:40111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=45483
--tserver_master_addrs=127.16.235.126:38477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:53.862548 3260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:53.862931 3260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:53.870093 3271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:53.870458 3275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:53.870715 3270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:53.870843 3260 server_base.cc:1034] running on GCE node
I20260806 22:39:53.871253 3260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:53.871614 3260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:53.872838 3260 hybrid_clock.cc:648] HybridClock initialized: now 1786055993872804 us; error 54 us; skew 500 ppm
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:53.876510 3260 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:39:53.877185 3260 webserver.cc:460] Webserver started at http://127.16.235.66:45483/ using document root <none> and password file <none>
I20260806 22:39:53.877504 3260 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:53.877619 3260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:53.899163 3260 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260806 22:39:53.900257 3294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.900494 3260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:53.900607 3260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "db3ca75bb94d41eea389485f3bcbb19d"
format_stamp: "Formatted at 2026-08-06 22:39:52 on dist-test-slave-3kk6"
server_key: "b12fd1190efe557543e0b7d33a021c1c"
server_key_iv: "d817c900192250473c33444d7a790473"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:53.901051 3260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:53.921465 3260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:53.923924 3260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:53.924113 3260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:53.924410 3260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:53.924762 3260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:53.924819 3260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.924866 3260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:53.924896 3260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:53.931951 3260 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:40111
I20260806 22:39:53.932098 3460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:40111 every 8 connection(s)
I20260806 22:39:53.932467 3260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:39:53.939568 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 3260
I20260806 22:39:53.939937 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.235.84:55433/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:45227
--local_ip_for_outbound_sockets=127.16.235.67
--tserver_master_addrs=127.16.235.126:38477
--webserver_port=34447
--webserver_interface=127.16.235.67
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.235.84:55433/kms with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:39:53 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:53.947355 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:53.934388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:38477 (local address 127.16.235.66:58823)
0806 22:39:53.935195 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:53.935261 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:53.935971 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:53.936354 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:53.936360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:53.936657 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:53.936978 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:53.936987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.938076 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.938077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:53.939464 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:53.939468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.940077 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.940079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:53.940227 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:53.940929 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:53.940940 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:53.943315 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:53.945516 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:53.945526 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:53.945529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:53.945888 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:53.946126 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:53.946127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:53.946127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:53.946271 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:53.946573 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:53.946580 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:53.946809 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 22:39:53.946918 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:39:53.947097 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":109,"threads_started":1}
I20260806 22:39:53.948056 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:53.934797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:58823 (local address 127.16.235.126:38477)
0806 22:39:53.934848 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:53.934850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:53.935405 (+ 555us) server_negotiation.cc:407] Connection header received
0806 22:39:53.936127 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:53.936129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:53.936147 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:53.936228 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:53.937119 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.937950 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.939589 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:53.939897 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:53.939952 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:53.943461 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:53.943463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:53.943464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:53.943474 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:53.945388 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:53.945988 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:53.945989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:53.945989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:53.946057 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:53.946341 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:53.946342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:53.946342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:53.946486 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:53.946545 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:53.947910 (+ 1365us) server_negotiation.cc:299] Negotiation successful
0806 22:39:53.947992 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:39:53.948484 3462 heartbeater.cc:347] Connected to a master server at 127.16.235.126:38477
I20260806 22:39:53.948623 3462 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:53.948843 3462 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
W20260806 22:39:53.949579 3462 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:38477 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:39:53.949671 3462 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:53.949755 3462 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
I20260806 22:39:53.950062 3462 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:53.950143 3462 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
W20260806 22:39:53.950435 3462 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:39:54.067428 3466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:54.067775 3466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:54.067827 3466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:54.069841 3466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:54.069921 3466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:54.069967 3466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:39:54.072654 3466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.235.84:55433/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:45227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=34447
--tserver_master_addrs=127.16.235.126:38477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:54.072962 3466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:54.073272 3466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:54.079165 3479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:54.079164 3475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:54.079480 3466 server_base.cc:1034] running on GCE node
W20260806 22:39:54.079593 3476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:54.079803 3466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:54.080102 3466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:54.081292 3466 hybrid_clock.cc:648] HybridClock initialized: now 1786055994081267 us; error 46 us; skew 500 ppm
Aug 06 22:39:54 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055994, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:54.084391 3466 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:39:54.084939 3466 webserver.cc:460] Webserver started at http://127.16.235.67:34447/ using document root <none> and password file <none>
I20260806 22:39:54.085207 3466 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:54.085300 3466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:39:54 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055994, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:54.109737 3466 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:54.110991 3491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:54.111184 3466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:54.111294 3466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "9713f837fbed4741a711b7e1e049ae81"
format_stamp: "Formatted at 2026-08-06 22:39:53 on dist-test-slave-3kk6"
server_key: "2ba707c4edfee315b1c5d893775b0530"
server_key_iv: "2b83fff7897e6ae9f51304118eac98d4"
server_key_version: "kuduclusterkey@0"
I20260806 22:39:54.111683 3466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:54.139551 3466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:54.142304 3466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:54.142499 3466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:54.142851 3466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:39:54.143253 3466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:39:54.143307 3466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:54.143347 3466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:39:54.143364 3466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:54.150552 3466 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:45227
I20260806 22:39:54.151394 3466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:39:54.152311 3654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:45227 every 8 connection(s)
I20260806 22:39:54.152745 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 3466
Aug 06 22:39:54 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055994, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:54.162629 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:54.153277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.16.235.126:38477)
0806 22:39:54.153384 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:54.153386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:54.154084 (+ 698us) server_negotiation.cc:407] Connection header received
0806 22:39:54.154140 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:54.154141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:54.154153 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:54.154242 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:54.154957 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:54.155694 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:54.157465 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:54.157677 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:54.157771 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:54.159398 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:54.159399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:54.159399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:54.159407 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:54.161226 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:54.161608 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:54.161609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:54.161610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:54.161675 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:54.161891 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:54.161892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:54.161893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:54.162014 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:54.162229 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:54.162398 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 22:39:54.162495 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260806 22:39:54.163730 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:54.152525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:48059 (local address 127.16.235.126:38477)
0806 22:39:54.152599 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:54.152601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:54.153231 (+ 630us) server_negotiation.cc:407] Connection header received
0806 22:39:54.153870 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:54.153871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:54.153885 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:54.153952 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:54.154873 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:54.155409 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:54.156629 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:54.156907 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:54.156980 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:54.159634 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:54.159635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:54.159635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:54.159643 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:54.161540 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:54.162078 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:54.162079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:54.162079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:54.162132 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:54.162472 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:54.162473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:54.162473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:54.162611 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:54.162725 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:54.163593 (+ 868us) server_negotiation.cc:299] Negotiation successful
0806 22:39:54.163664 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:39:54.164021 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:54.152372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:38477 (local address 127.16.235.67:48059)
0806 22:39:54.153057 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:54.153103 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:54.153750 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:54.154098 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:54.154103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:54.154395 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:54.154756 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:54.154763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:54.155524 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:54.155525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:54.156530 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:54.156533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:54.157033 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:54.157035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:54.157183 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:54.157980 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:54.157989 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:54.159526 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:54.161648 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:54.161658 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:54.161661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:54.161992 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:54.162209 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:54.162210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:54.162210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:54.162376 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:54.162756 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:54.162764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:54.162946 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 22:39:54.163578 (+ 632us) client_negotiation.cc:240] Negotiation successful
0806 22:39:54.163772 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":578,"threads_started":1}
I20260806 22:39:54.164525 3655 heartbeater.cc:347] Connected to a master server at 127.16.235.126:38477
I20260806 22:39:54.164646 3655 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:54.164891 3655 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
W20260806 22:39:54.165604 3655 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:38477 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:39:54.165691 3655 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:54.165814 3655 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
I20260806 22:39:54.166306 3655 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:54.166401 3655 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
W20260806 22:39:54.166814 3655 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 22:39:54 dist-test-slave-3kk6 krb5kdc[1533](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786055994, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:54.753126 3256 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:54.753319 3256 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
I20260806 22:39:54.753841 2957 ts_manager.cc:194] Registered new tserver with Master: 574dfa6705224e98ae24cdf61b18e1cf (127.16.235.65:33675)
I20260806 22:39:54.754621 2957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:56497
I20260806 22:39:54.950696 3462 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:54.950883 3462 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
I20260806 22:39:54.951203 2957 ts_manager.cc:194] Registered new tserver with Master: db3ca75bb94d41eea389485f3bcbb19d (127.16.235.66:40111)
I20260806 22:39:54.951774 2957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:58823
Aug 06 22:39:55 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055994, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:55.166996 3655 heartbeater.cc:464] Registering TS with master...
I20260806 22:39:55.167186 3655 heartbeater.cc:511] Master 127.16.235.126:38477 requested a full tablet report, sending...
I20260806 22:39:55.167568 2957 ts_manager.cc:194] Registered new tserver with Master: 9713f837fbed4741a711b7e1e049ae81 (127.16.235.67:45227)
I20260806 22:39:55.167990 2957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:48059
I20260806 22:39:55.170382 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:39:55.181769 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.171546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.16.235.126:38477)
0806 22:39:55.174774 (+ 3228us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:55.174776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:55.174785 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:39:55.174827 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:55.174827 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:55.174838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:55.174936 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:55.175685 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.176379 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.177121 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.177352 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.177434 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.178925 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:55.178927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:55.178927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:55.178935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:55.180545 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:55.180935 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:55.180936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:55.180937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:55.181002 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:55.181214 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:55.181215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:55.181215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:55.181337 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:55.181444 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:55.181554 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:39:55.181648 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3204,"thread_start_us":73,"threads_started":1}
I20260806 22:39:55.184221 2957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:39:55.186048 2957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:39:55.207119 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.200651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:40111 (local address 127.0.0.1:35834)
0806 22:39:55.201802 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:55.201846 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:55.201918 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:55.202253 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:55.202257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:55.202259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:55.202340 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.202342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.204101 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.204102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.205443 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.205446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.206815 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.206816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.206891 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.206905 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:39:55.206976 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:39:55.207026 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1108,"thread_start_us":125,"threads_started":1}
I20260806 22:39:55.207414 3697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.200970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.16.235.66:40111)
0806 22:39:55.201922 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:55.201926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:55.201939 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:39:55.202045 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:55.202047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:55.202067 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:55.202141 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:55.202431 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.204017 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.205833 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.206592 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.206692 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.206766 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:55.207268 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 22:39:55.207319 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":917,"thread_start_us":62,"threads_started":1}
I20260806 22:39:55.208335 3366 tablet_service.cc:1467] Processing CreateTablet for tablet 23c6ee8979de4258b4d0ad2fc48f9716 (DEFAULT_TABLE table=test-table [id=bed0cc44886d4be9a1241e124f890c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:55.208842 3366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c6ee8979de4258b4d0ad2fc48f9716. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:55.209345 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.200651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:33675 (local address 127.0.0.1:38442)
0806 22:39:55.201989 (+ 1338us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:55.201999 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:55.202083 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:55.202373 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:55.202374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:55.202376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:55.202425 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.202427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.204064 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.204065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.206620 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.206623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.209073 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.209074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.209154 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.209158 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:39:55.209208 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:39:55.209248 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1323,"mutex_wait_us":2,"thread_start_us":121,"threads_started":1}
I20260806 22:39:55.209445 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.200938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:45227 (local address 127.0.0.1:55534)
0806 22:39:55.202516 (+ 1578us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:55.202526 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:55.202569 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:55.204171 (+ 1602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:55.204172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:55.204175 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:55.204229 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.204230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.206722 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.206723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.208182 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.208186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.209215 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.209216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.209269 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.209272 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:39:55.209332 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:39:55.209371 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1561,"thread_start_us":77,"threads_started":1}
I20260806 22:39:55.209638 3698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.201162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.16.235.65:33675)
0806 22:39:55.201674 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:55.201678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:55.202056 (+ 378us) server_negotiation.cc:407] Connection header received
0806 22:39:55.202176 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:55.202178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:55.202191 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:55.202243 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:55.202913 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.203966 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.207807 (+ 3841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.208900 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.208951 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.209004 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:55.209510 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 22:39:55.209564 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":68,"threads_started":1}
I20260806 22:39:55.209719 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.201252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.16.235.67:45227)
0806 22:39:55.201468 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:55.201473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:55.202630 (+ 1157us) server_negotiation.cc:407] Connection header received
0806 22:39:55.202705 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:55.202708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:55.202725 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:55.202801 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:55.204325 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.205455 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.208280 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.209069 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.209131 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.209183 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:55.209589 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 22:39:55.209632 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":87,"threads_started":1}
I20260806 22:39:55.210254 3165 tablet_service.cc:1467] Processing CreateTablet for tablet 23c6ee8979de4258b4d0ad2fc48f9716 (DEFAULT_TABLE table=test-table [id=bed0cc44886d4be9a1241e124f890c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:55.210541 3165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c6ee8979de4258b4d0ad2fc48f9716. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:55.211180 3554 tablet_service.cc:1467] Processing CreateTablet for tablet 23c6ee8979de4258b4d0ad2fc48f9716 (DEFAULT_TABLE table=test-table [id=bed0cc44886d4be9a1241e124f890c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:39:55.211619 3554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c6ee8979de4258b4d0ad2fc48f9716. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:55.213397 3700 tablet_bootstrap.cc:492] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: Bootstrap starting.
I20260806 22:39:55.214221 3700 tablet_bootstrap.cc:654] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:55.214615 3700 log.cc:824] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:55.215171 3703 tablet_bootstrap.cc:492] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf: Bootstrap starting.
I20260806 22:39:55.215888 3703 tablet_bootstrap.cc:654] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:55.216441 3703 log.cc:824] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:55.216701 3704 tablet_bootstrap.cc:492] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: Bootstrap starting.
I20260806 22:39:55.217443 3704 tablet_bootstrap.cc:654] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:55.217752 3704 log.cc:824] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:55.218708 3700 tablet_bootstrap.cc:492] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: No bootstrap required, opened a new log
I20260806 22:39:55.218811 3700 ts_tablet_manager.cc:1397] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:39:55.219244 3704 tablet_bootstrap.cc:492] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: No bootstrap required, opened a new log
I20260806 22:39:55.219388 3704 ts_tablet_manager.cc:1397] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:39:55.221671 3700 raft_consensus.cc:348] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } }
I20260806 22:39:55.221889 3700 raft_consensus.cc:374] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:55.221943 3700 raft_consensus.cc:729] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3ca75bb94d41eea389485f3bcbb19d, State: Initialized, Role: FOLLOWER
I20260806 22:39:55.222090 3700 consensus_queue.cc:260] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } }
I20260806 22:39:55.222476 3700 ts_tablet_manager.cc:1428] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260806 22:39:55.222612 3462 heartbeater.cc:503] Master 127.16.235.126:38477 was elected leader, sending a full tablet report...
I20260806 22:39:55.222940 3704 raft_consensus.cc:348] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } }
I20260806 22:39:55.223115 3704 raft_consensus.cc:374] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:55.223147 3704 raft_consensus.cc:729] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9713f837fbed4741a711b7e1e049ae81, State: Initialized, Role: FOLLOWER
I20260806 22:39:55.223274 3704 consensus_queue.cc:260] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } }
I20260806 22:39:55.223588 3704 ts_tablet_manager.cc:1428] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 22:39:55.223742 3655 heartbeater.cc:503] Master 127.16.235.126:38477 was elected leader, sending a full tablet report...
I20260806 22:39:55.224545 3703 tablet_bootstrap.cc:492] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf: No bootstrap required, opened a new log
I20260806 22:39:55.224629 3703 ts_tablet_manager.cc:1397] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 22:39:55.227975 3703 raft_consensus.cc:348] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } }
I20260806 22:39:55.228188 3703 raft_consensus.cc:374] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:55.228237 3703 raft_consensus.cc:729] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 574dfa6705224e98ae24cdf61b18e1cf, State: Initialized, Role: FOLLOWER
I20260806 22:39:55.228399 3703 consensus_queue.cc:260] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } }
I20260806 22:39:55.228786 3703 ts_tablet_manager.cc:1428] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:39:55.228924 3256 heartbeater.cc:503] Master 127.16.235.126:38477 was elected leader, sending a full tablet report...
W20260806 22:39:55.229045 3258 tablet.cc:2369] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:39:55.293217 3708 raft_consensus.cc:482] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:39:55.293467 3708 raft_consensus.cc:504] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } }
I20260806 22:39:55.294040 3708 leader_election.cc:288] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 574dfa6705224e98ae24cdf61b18e1cf (127.16.235.65:33675), 9713f837fbed4741a711b7e1e049ae81 (127.16.235.67:45227)
I20260806 22:39:55.298547 3698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.294413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:33285 (local address 127.16.235.65:33675)
0806 22:39:55.294458 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:55.294460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:55.294686 (+ 226us) server_negotiation.cc:407] Connection header received
0806 22:39:55.294816 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:55.294817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:55.294827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:55.294860 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:55.295122 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.295895 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.297402 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.298106 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.298165 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.298176 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:55.298425 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:39:55.298471 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:39:55.298599 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.294723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:35413 (local address 127.16.235.67:45227)
0806 22:39:55.294769 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:55.294771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:55.294885 (+ 114us) server_negotiation.cc:407] Connection header received
0806 22:39:55.295005 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:55.295006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:55.295016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:55.295047 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:39:55.295405 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.296161 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.297552 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.298104 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.298165 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.298176 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:55.298479 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 22:39:55.298529 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:39:55.298918 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.294539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:45227 (local address 127.16.235.66:35413)
0806 22:39:55.294823 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:55.294833 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:55.294924 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:55.295123 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:55.295124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:55.295125 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:55.295300 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.295302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.296278 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.296279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.297432 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.297436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.298275 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.298276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.298377 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.298381 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:39:55.298762 (+ 381us) client_negotiation.cc:240] Negotiation successful
0806 22:39:55.298814 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":61,"threads_started":1}
I20260806 22:39:55.299280 3558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23c6ee8979de4258b4d0ad2fc48f9716" candidate_uuid: "db3ca75bb94d41eea389485f3bcbb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9713f837fbed4741a711b7e1e049ae81" is_pre_election: true
I20260806 22:39:55.299467 3558 raft_consensus.cc:2454] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3ca75bb94d41eea389485f3bcbb19d in term 0.
I20260806 22:39:55.299619 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.294311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:33675 (local address 127.16.235.66:33285)
0806 22:39:55.294584 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:55.294597 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:55.294719 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:55.294948 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:55.294949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:55.294952 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:39:55.295038 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.295041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.296007 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.296008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.297283 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:55.297286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.298264 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.298265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:55.298336 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.298340 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:39:55.299210 (+ 870us) client_negotiation.cc:240] Negotiation successful
0806 22:39:55.299271 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":126,"threads_started":1}
I20260806 22:39:55.299736 3299 leader_election.cc:302] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9713f837fbed4741a711b7e1e049ae81, db3ca75bb94d41eea389485f3bcbb19d; no voters:
I20260806 22:39:55.299943 3708 raft_consensus.cc:2781] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:39:55.300017 3708 raft_consensus.cc:482] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:39:55.300045 3708 raft_consensus.cc:3037] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:55.300233 3186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23c6ee8979de4258b4d0ad2fc48f9716" candidate_uuid: "db3ca75bb94d41eea389485f3bcbb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "574dfa6705224e98ae24cdf61b18e1cf" is_pre_election: true
I20260806 22:39:55.300397 3186 raft_consensus.cc:2454] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3ca75bb94d41eea389485f3bcbb19d in term 0.
I20260806 22:39:55.301239 3708 raft_consensus.cc:504] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } }
I20260806 22:39:55.301388 3708 leader_election.cc:288] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [CANDIDATE]: Term 1 election: Requested vote from peers 574dfa6705224e98ae24cdf61b18e1cf (127.16.235.65:33675), 9713f837fbed4741a711b7e1e049ae81 (127.16.235.67:45227)
I20260806 22:39:55.301608 3186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23c6ee8979de4258b4d0ad2fc48f9716" candidate_uuid: "db3ca75bb94d41eea389485f3bcbb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "574dfa6705224e98ae24cdf61b18e1cf"
I20260806 22:39:55.301640 3558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23c6ee8979de4258b4d0ad2fc48f9716" candidate_uuid: "db3ca75bb94d41eea389485f3bcbb19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9713f837fbed4741a711b7e1e049ae81"
I20260806 22:39:55.301683 3186 raft_consensus.cc:3037] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:55.301723 3558 raft_consensus.cc:3037] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:55.302601 3186 raft_consensus.cc:2454] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3ca75bb94d41eea389485f3bcbb19d in term 1.
I20260806 22:39:55.302678 3558 raft_consensus.cc:2454] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3ca75bb94d41eea389485f3bcbb19d in term 1.
I20260806 22:39:55.302874 3299 leader_election.cc:302] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9713f837fbed4741a711b7e1e049ae81, db3ca75bb94d41eea389485f3bcbb19d; no voters:
I20260806 22:39:55.303031 3708 raft_consensus.cc:2781] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:55.303229 3708 raft_consensus.cc:686] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 1 LEADER]: Becoming Leader. State: Replica: db3ca75bb94d41eea389485f3bcbb19d, State: Running, Role: LEADER
I20260806 22:39:55.303331 3708 consensus_queue.cc:237] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } }
I20260806 22:39:55.304188 2957 catalog_manager.cc:5495] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d reported cstate change: term changed from 0 to 1, leader changed from <none> to db3ca75bb94d41eea389485f3bcbb19d (127.16.235.66). New cstate: current_term: 1 leader_uuid: "db3ca75bb94d41eea389485f3bcbb19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db3ca75bb94d41eea389485f3bcbb19d" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 } health_report { overall_health: UNKNOWN } } }
I20260806 22:39:55.323400 3697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.319819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35846 (local address 127.16.235.66:40111)
0806 22:39:55.319873 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:55.319876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:55.319886 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:39:55.320019 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:55.320021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:55.320048 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:55.320149 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:39:55.320377 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.321119 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.322167 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:55.322533 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:55.322620 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:55.322841 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:39:55.322984 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:39:55.323061 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:55.323296 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 22:39:55.323321 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:39:55.327982 3558 raft_consensus.cc:1261] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [term 1 FOLLOWER]: Refusing update from remote peer db3ca75bb94d41eea389485f3bcbb19d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:55.328368 3186 raft_consensus.cc:1261] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf [term 1 FOLLOWER]: Refusing update from remote peer db3ca75bb94d41eea389485f3bcbb19d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:55.328855 3723 consensus_queue.cc:1035] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9713f837fbed4741a711b7e1e049ae81" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 45227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:55.329064 3723 consensus_queue.cc:1035] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "574dfa6705224e98ae24cdf61b18e1cf" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:55.334702 3726 mvcc.cc:205] Tried to move back new op lower bound from 7315685356859666432 to 7315685356762980352. Current Snapshot: MvccSnapshot[applied={T|T < 7315685356859666432}]
I20260806 22:39:55.335489 3731 mvcc.cc:205] Tried to move back new op lower bound from 7315685356859666432 to 7315685356762980352. Current Snapshot: MvccSnapshot[applied={T|T < 7315685356859666432}]
I20260806 22:39:55.340288 2957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:55.340407 2957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:39:55.341583 2957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52b939a2e69e4a9d967d2c0d5ecc521c: Sending DeleteTablet for 3 replicas of tablet 23c6ee8979de4258b4d0ad2fc48f9716
I20260806 22:39:55.342155 3165 tablet_service.cc:1514] Processing DeleteTablet for tablet 23c6ee8979de4258b4d0ad2fc48f9716 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:55 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:38442
I20260806 22:39:55.342417 3554 tablet_service.cc:1514] Processing DeleteTablet for tablet 23c6ee8979de4258b4d0ad2fc48f9716 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:55 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:55534
I20260806 22:39:55.342638 3366 tablet_service.cc:1514] Processing DeleteTablet for tablet 23c6ee8979de4258b4d0ad2fc48f9716 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:39:55 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35834
I20260806 22:39:55.343493 3740 tablet_replica.cc:330] T 23c6ee8979de4258b4d0ad2fc48f9716 P 574dfa6705224e98ae24cdf61b18e1cf: stopping tablet replica
I20260806 22:39:55.343560 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 3014
I20260806 22:39:55.343716 3744 tablet_replica.cc:330] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: stopping tablet replica
I20260806 22:39:55.343851 3741 tablet_replica.cc:330] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: stopping tablet replica
I20260806 22:39:55.343992 3744 raft_consensus.cc:2229] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:39:55.344120 3741 raft_consensus.cc:2229] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:39:55.344322 3744 raft_consensus.cc:2258] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:55.344476 3741 raft_consensus.cc:2258] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:39:55.346825 3744 ts_tablet_manager.cc:1905] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:55.350945 3744 ts_tablet_manager.cc:1918] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:55.351729 3741 ts_tablet_manager.cc:1905] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:39:55.353477 3741 ts_tablet_manager.cc:1918] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:39:55.353641 3741 log.cc:1196] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/23c6ee8979de4258b4d0ad2fc48f9716
I20260806 22:39:55.353981 3741 ts_tablet_manager.cc:1939] T 23c6ee8979de4258b4d0ad2fc48f9716 P 9713f837fbed4741a711b7e1e049ae81: Deleting consensus metadata
I20260806 22:39:55.354584 2936 catalog_manager.cc:4841] TS 9713f837fbed4741a711b7e1e049ae81 (127.16.235.67:45227): tablet 23c6ee8979de4258b4d0ad2fc48f9716 (table test-table [id=bed0cc44886d4be9a1241e124f890c00]) successfully deleted
I20260806 22:39:55.354817 3744 log.cc:1196] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/23c6ee8979de4258b4d0ad2fc48f9716
I20260806 22:39:55.355193 3744 ts_tablet_manager.cc:1939] T 23c6ee8979de4258b4d0ad2fc48f9716 P db3ca75bb94d41eea389485f3bcbb19d: Deleting consensus metadata
I20260806 22:39:55.355720 2937 catalog_manager.cc:4841] TS db3ca75bb94d41eea389485f3bcbb19d (127.16.235.66:40111): tablet 23c6ee8979de4258b4d0ad2fc48f9716 (table test-table [id=bed0cc44886d4be9a1241e124f890c00]) successfully deleted
W20260806 22:39:55.356614 2936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:33675 (error 108)
I20260806 22:39:55.357772 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:55.357610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:33675 (local address 127.0.0.1:38458)
0806 22:39:55.357667 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:55.357738 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:33675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 22:39:55.357821 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 3260
W20260806 22:39:55.357913 2936 catalog_manager.cc:4584] TS 574dfa6705224e98ae24cdf61b18e1cf (127.16.235.65:33675): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23c6ee8979de4258b4d0ad2fc48f9716: Network error: Client connection negotiation failed: client connection to 127.16.235.65:33675: connect: Connection refused (error 111)
I20260806 22:39:55.377398 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 3466
Aug 06 22:39:55 dist-test-slave-3kk6 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055994, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for HTTP/127.16.235.84@KRBTEST.COM
I20260806 22:39:55.397847 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 2904
I20260806 22:39:55.412223 17325 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:39:55.767480 17325 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:39:55.767588 17325 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:39:56.122790 17325 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:39:56.123 UTC [1578] LOG: received smart shutdown request
2026-08-06 22:39:56.128 UTC [1578] LOG: background worker "logical replication launcher" (PID 1591) exited with exit code 1
2026-08-06 22:39:56.128 UTC [1582] LOG: shutting down
2026-08-06 22:39:56.303 UTC [1578] LOG: database system is shut down
2026-08-06T22:39:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50923 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:56 dist-test-slave-3kk6 krb5kdc[3765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:56 dist-test-slave-3kk6 krb5kdc[3765](info): set up 2 sockets
Aug 06 22:39:56 dist-test-slave-3kk6 krb5kdc[3765](info): commencing operation
krb5kdc: starting...
W20260806 22:39:57.640511 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:39:57 dist-test-slave-3kk6 krb5kdc[3765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:39:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:39:57.777956 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33433
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:33433,127.16.235.125:34723,127.16.235.124:44195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:39:57.866231 3783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:57.866546 3783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:57.866598 3783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:57.867875 3783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:57.867973 3783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:57.867998 3783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:57.868019 3783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:57.868052 3783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:57.869709 3783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:33433,127.16.235.125:34723,127.16.235.124:44195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:57.870038 3783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:57.870297 3783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:57.875480 3790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:57.875474 3792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:57.875790 3783 server_base.cc:1034] running on GCE node
W20260806 22:39:57.875474 3794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:57.876093 3783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:57.876408 3783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:57.877579 3783 hybrid_clock.cc:648] HybridClock initialized: now 1786055997877574 us; error 38 us; skew 500 ppm
Aug 06 22:39:57 dist-test-slave-3kk6 krb5kdc[3765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:57.884232 3783 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:39:57.884796 3783 webserver.cc:460] Webserver started at http://127.16.235.126:37035/ using document root <none> and password file <none>
I20260806 22:39:57.885043 3783 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:57.885116 3783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:57.885284 3783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:57.886463 3783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "e12f416a52b841889e96699402a7db50"
format_stamp: "Formatted at 2026-08-06 22:39:57 on dist-test-slave-3kk6"
server_key: "085a92a83b54e353c8872eb8b77c3dcc"
server_key_iv: "604aa20e7bb573a9e1f5ec9f6379d8e8"
server_key_version: "encryptionkey@0"
I20260806 22:39:57.886893 3783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "e12f416a52b841889e96699402a7db50"
format_stamp: "Formatted at 2026-08-06 22:39:57 on dist-test-slave-3kk6"
server_key: "085a92a83b54e353c8872eb8b77c3dcc"
server_key_iv: "604aa20e7bb573a9e1f5ec9f6379d8e8"
server_key_version: "encryptionkey@0"
I20260806 22:39:57.888331 3783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:39:57.889066 3802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:57.889245 3783 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:57.889359 3783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "e12f416a52b841889e96699402a7db50"
format_stamp: "Formatted at 2026-08-06 22:39:57 on dist-test-slave-3kk6"
server_key: "085a92a83b54e353c8872eb8b77c3dcc"
server_key_iv: "604aa20e7bb573a9e1f5ec9f6379d8e8"
server_key_version: "encryptionkey@0"
I20260806 22:39:57.889443 3783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:57.908952 3783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:57.911626 3783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:57.911840 3783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:57.916414 3783 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:33433
I20260806 22:39:57.916456 3886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:33433 every 8 connection(s)
I20260806 22:39:57.916872 3783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:39:57.917805 3887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:57.919519 3887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:39:57.925071 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 3783
I20260806 22:39:57.925228 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:39:57.925475 17325 external_mini_cluster.cc:1428] Setting key 2270b882117ec979e2ad04929d5617e6
Aug 06 22:39:57 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:57.931103 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:57.920309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.16.235.126:33433)
0806 22:39:57.920504 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:57.920513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:57.920644 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:39:57.921210 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:57.921212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:57.921237 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:57.921335 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:57.921894 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:57.922840 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:57.923763 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:57.924098 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:57.924226 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:57.926870 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:57.926878 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:57.926878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:57.926886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:57.929143 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:57.929650 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:57.929653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:57.929654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:57.929746 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:57.930019 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:57.930020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:57.930020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:57.930264 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:57.930334 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:57.930613 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 22:39:57.930797 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260806 22:39:57.931136 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:57.920285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33433 (local address 127.0.0.1:41726)
0806 22:39:57.920493 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:57.920540 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:57.921048 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:57.921341 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:57.921343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:57.921623 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:57.921793 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:57.921798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:57.922967 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:57.922969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:57.923668 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:57.923670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:57.924184 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:57.924185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:57.924259 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:57.924811 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:57.924821 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:57.926774 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:57.929243 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:57.929252 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:57.929254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:57.929560 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:57.929821 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:57.929822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:57.929822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:57.929947 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:57.930352 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:57.930360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:57.930455 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 22:39:57.930583 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:39:57.930920 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":110,"thread_start_us":90,"threads_started":1}
I20260806 22:39:57.932026 3887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:39:57.932487 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:57.932325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:34723 (local address 127.0.0.1:36630)
0806 22:39:57.932388 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:57.932452 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 22:39:57.932657 3806 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
I20260806 22:39:57.933192 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:57.933077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:34723 (local address 127.0.0.1:36638)
0806 22:39:57.933132 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:57.933168 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:39:57.933368 3887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.125:34723: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.16.235.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab.
Entry for principal kudu/127.16.235.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab.
I20260806 22:39:57.954262 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.125:34723
--webserver_interface=127.16.235.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab
--principal=kudu/127.16.235.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:33433,127.16.235.125:34723,127.16.235.124:44195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:39:57.982676 3887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } attempt: 1
I20260806 22:39:57.983354 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:57.983155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:34723 (local address 127.0.0.1:36640)
0806 22:39:57.983241 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:57.983296 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 22:39:57.983795 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:57.983668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:34723 (local address 127.0.0.1:36652)
0806 22:39:57.983734 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:57.983769 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 22:39:57.983949 3887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.125:34723: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
W20260806 22:39:58.040784 3897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:58.041033 3897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:58.041059 3897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:58.042447 3897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:58.042496 3897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:58.042510 3897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:58.042524 3897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:58.042577 3897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:58.044158 3897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:33433,127.16.235.125:34723,127.16.235.124:44195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab
--principal=kudu/127.16.235.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.125:34723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.235.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:58.044437 3897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:58.044693 3897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:58.049966 3905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:58.050006 3897 server_base.cc:1034] running on GCE node
W20260806 22:39:58.050011 3906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:58.049962 3908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:58.050542 3897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:58.050901 3897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:58.052102 3897 hybrid_clock.cc:648] HybridClock initialized: now 1786055998052075 us; error 38 us; skew 500 ppm
I20260806 22:39:58.052111 3887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } attempt: 2
I20260806 22:39:58.052547 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.052427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:34723 (local address 127.0.0.1:36656)
0806 22:39:58.052487 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.052521 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:39:58.052883 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.052784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:34723 (local address 127.0.0.1:36658)
0806 22:39:58.052831 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.052855 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:39:58.053050 3887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.125:34723: Network error: Client connection negotiation failed: client connection to 127.16.235.125:34723: connect: Connection refused (error 111)
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:58.055040 3897 init.cc:377] Logged in from keytab as kudu/127.16.235.125@KRBTEST.COM (short username kudu)
I20260806 22:39:58.055501 3897 webserver.cc:460] Webserver started at http://127.16.235.125:37879/ using document root <none> and password file <none>
I20260806 22:39:58.055734 3897 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:58.055783 3897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:58.055935 3897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:58.057058 3897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/data/instance:
uuid: "1f14ad9fafa84403a88edf8390b3474e"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-3kk6"
server_key: "39d818f9e5036a57a6838e7cfd813aae"
server_key_iv: "20ad76e577d4b9cd2d4afb9ff7b0e7f1"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.057442 3897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/wal/instance:
uuid: "1f14ad9fafa84403a88edf8390b3474e"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-3kk6"
server_key: "39d818f9e5036a57a6838e7cfd813aae"
server_key_iv: "20ad76e577d4b9cd2d4afb9ff7b0e7f1"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.059108 3897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:39:58.059991 3921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:58.060197 3897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:39:58.060307 3897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/wal
uuid: "1f14ad9fafa84403a88edf8390b3474e"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-3kk6"
server_key: "39d818f9e5036a57a6838e7cfd813aae"
server_key_iv: "20ad76e577d4b9cd2d4afb9ff7b0e7f1"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.060386 3897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:58.070293 3897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:58.072532 3897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:58.072708 3897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:58.077184 3897 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.125:34723
I20260806 22:39:58.077257 3992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.125:34723 every 8 connection(s)
I20260806 22:39:58.077631 3897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
I20260806 22:39:58.078533 3993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:58.080444 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 3897
I20260806 22:39:58.080591 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-1/wal/instance
I20260806 22:39:58.080456 3993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:39:58.080789 17325 external_mini_cluster.cc:1428] Setting key 13f232d3cf29407d8ca9a456d7ab1084
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.235.124@KRBTEST.COM; defaulting to no policy
I20260806 22:39:58.090955 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.081406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.16.235.126:33433)
0806 22:39:58.081470 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.081472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.081685 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:39:58.082207 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.082209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.082222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.082288 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.083253 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.083862 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.084864 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.085255 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.085340 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.087644 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.087646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.087647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.087657 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.089479 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.089958 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.089960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.089960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.090028 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.090278 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.090279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.090279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.090399 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.090456 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.090787 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.090883 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:39:58.091241 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.081375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33433 (local address 127.0.0.1:45558)
0806 22:39:58.081556 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.081593 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:58.082090 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:58.082374 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:58.082378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:58.082832 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:58.083139 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.083146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.083967 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.083969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.084766 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.084769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.085365 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.085367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.085490 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.086081 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:58.086090 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:58.087527 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:58.089576 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.089583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.089585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.089825 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.090099 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.090099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.090100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.090209 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.090462 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:58.090467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:58.090593 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:39:58.090748 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 22:39:58.090970 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260806 22:39:58.091858 3993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.124@KRBTEST.COM" created.
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
I20260806 22:39:58.101202 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.092159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:34723 (local address 127.0.0.1:55250)
0806 22:39:58.092350 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.092365 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:58.092415 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:58.092612 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:58.092613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:58.092787 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:58.092852 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.092853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.093827 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.093828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.094341 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.094343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.094925 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.094926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.094999 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.095524 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:58.095530 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:58.097435 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:58.099993 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.099994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.099994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.100223 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.100439 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.100440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.100440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.100493 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.100864 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:58.100865 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:58.100922 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 22:39:58.100995 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:39:58.101099 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260806 22:39:58.101534 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.092234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55250 (local address 127.16.235.125:34723)
0806 22:39:58.092453 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.092459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.092483 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:39:58.092504 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.092506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.092519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.092612 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.092931 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.093721 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.094442 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.094832 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.094926 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.097542 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.097551 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.097552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.097559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.099893 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.100315 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.100319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.100319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.100378 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.100549 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.100550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.100550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.100740 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.100826 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.101352 (+ 526us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.101437 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":141,"threads_started":1}
I20260806 22:39:58.102474 3993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:39:58.102916 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.102798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52638)
0806 22:39:58.102837 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.102890 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 22:39:58.103259 3925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
I20260806 22:39:58.104056 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.103958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52640)
0806 22:39:58.104003 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.104032 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 22:39:58.104238 3993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:44195: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab.
Entry for principal kudu/127.16.235.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab.
I20260806 22:39:58.113135 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.124:44195
--webserver_interface=127.16.235.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab
--principal=kudu/127.16.235.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:33433,127.16.235.125:34723,127.16.235.124:44195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:39:58.144236 3887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } attempt: 3
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
I20260806 22:39:58.151787 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.144727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:34723 (local address 127.0.0.1:55266)
0806 22:39:58.144816 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.144851 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:58.144951 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:58.145123 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:58.145124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:58.145317 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:58.145393 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.145395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.146099 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.146100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.146615 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.146617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.147069 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.147070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.147172 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.147694 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:58.147699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:58.148988 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:58.150605 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.150606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.150607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.150841 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.151065 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.151066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.151066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.151134 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.151405 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:58.151407 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:58.151459 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:39:58.151551 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:39:58.151679 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 22:39:58.151787 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.144838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.16.235.125:34723)
0806 22:39:58.144913 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.144915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.144928 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:39:58.145027 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.145029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.145046 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.145115 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.145488 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.146005 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.146760 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.146989 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.147071 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.149083 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.149085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.149085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.149094 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.150513 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.150922 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.150923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.150923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.150988 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.151207 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.151208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.151208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.151327 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.151414 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.151577 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.151683 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:39:58.152456 3887 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:39:58.152884 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.152765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52642)
0806 22:39:58.152821 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.152861 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:39:58.153204 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.153115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52648)
0806 22:39:58.153159 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.153184 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:39:58.153301 3887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:44195: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
I20260806 22:39:58.153460 3993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } attempt: 1
I20260806 22:39:58.153798 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.153690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52656)
0806 22:39:58.153742 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.153771 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:39:58.154101 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.154007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52672)
0806 22:39:58.154056 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.154079 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:39:58.154196 3993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:44195: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
I20260806 22:39:58.184448 3887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } attempt: 1
I20260806 22:39:58.184998 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.184845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52686)
0806 22:39:58.184930 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.184967 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 22:39:58.185410 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.185312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52694)
0806 22:39:58.185364 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.185391 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 22:39:58.185509 3887 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:44195: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
W20260806 22:39:58.201601 4009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:39:58.201854 4009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:39:58.201881 4009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:39:58.203168 4009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:39:58.203220 4009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:39:58.203235 4009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:39:58.203248 4009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:39:58.203260 4009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:39:58.204825 4009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:33433,127.16.235.125:34723,127.16.235.124:44195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab
--principal=kudu/127.16.235.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.124:44195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.235.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:39:58.205109 4009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:39:58.205369 4009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:39:58.210731 4014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:58.210747 4019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:39:58.210747 4015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:39:58.210820 4009 server_base.cc:1034] running on GCE node
I20260806 22:39:58.211225 4009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:39:58.211578 4009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:39:58.212764 4009 hybrid_clock.cc:648] HybridClock initialized: now 1786055998212769 us; error 49 us; skew 500 ppm
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:39:58.215545 4009 init.cc:377] Logged in from keytab as kudu/127.16.235.124@KRBTEST.COM (short username kudu)
I20260806 22:39:58.216099 4009 webserver.cc:460] Webserver started at http://127.16.235.124:34133/ using document root <none> and password file <none>
I20260806 22:39:58.216363 4009 fs_manager.cc:359] Metadata directory not provided
I20260806 22:39:58.216439 4009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:39:58.216598 4009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:39:58.217772 4009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/data/instance:
uuid: "e36e60bdb3c3438c86cff6917156c361"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-3kk6"
server_key: "ed8b963c5ab94670ab7382f522f24e86"
server_key_iv: "3dfe66b2f80dc2567915a2a8333fbe79"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.218196 4009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/wal/instance:
uuid: "e36e60bdb3c3438c86cff6917156c361"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-3kk6"
server_key: "ed8b963c5ab94670ab7382f522f24e86"
server_key_iv: "3dfe66b2f80dc2567915a2a8333fbe79"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.219761 4009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:39:58.220592 4030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:39:58.220887 4009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:39:58.220997 4009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/wal
uuid: "e36e60bdb3c3438c86cff6917156c361"
format_stamp: "Formatted at 2026-08-06 22:39:58 on dist-test-slave-3kk6"
server_key: "ed8b963c5ab94670ab7382f522f24e86"
server_key_iv: "3dfe66b2f80dc2567915a2a8333fbe79"
server_key_version: "encryptionkey@0"
I20260806 22:39:58.221081 4009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:39:58.222335 3993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } attempt: 2
I20260806 22:39:58.222826 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.222706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52704)
0806 22:39:58.222752 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.222797 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 22:39:58.223192 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.223094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52710)
0806 22:39:58.223141 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.223166 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:39:58.223281 3993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:44195: Network error: Client connection negotiation failed: client connection to 127.16.235.124:44195: connect: Connection refused (error 111)
I20260806 22:39:58.231004 4009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:39:58.233587 4009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:39:58.233775 4009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:39:58.238276 4009 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.124:44195
I20260806 22:39:58.238314 4117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.124:44195 every 8 connection(s)
I20260806 22:39:58.238791 4009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
I20260806 22:39:58.239532 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 4009
I20260806 22:39:58.239673 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786055836003438-17325-0/minicluster-data/master-2/wal/instance
I20260806 22:39:58.239892 17325 external_mini_cluster.cc:1428] Setting key c7a1bc1670936c5a8159a8df08d864ac
I20260806 22:39:58.239977 4120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:39:58.242789 4120 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:58.251775 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.240523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.16.235.124:44195)
0806 22:39:58.240723 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.240731 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.240766 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:39:58.241213 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.241221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.241270 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.241387 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.242248 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.243419 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.244569 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.245186 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.245314 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.247993 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.248000 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.248001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.248007 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.250041 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.250411 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.250415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.250416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.250530 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.250791 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.250792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.250792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.251115 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.251278 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.251375 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.251523 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260806 22:39:58.251865 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.243588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45572 (local address 127.16.235.126:33433)
0806 22:39:58.243644 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.243646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.243797 (+ 151us) server_negotiation.cc:407] Connection header received
0806 22:39:58.243856 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.243857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.243867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.243917 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.244513 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.245235 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.246342 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.246532 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.246584 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.249009 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.249011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.249011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.249019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.250465 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.250956 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.250957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.250958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.251014 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.251234 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.251235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.251236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.251380 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.251451 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.251688 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.251769 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:39:58.251931 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.243481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33433 (local address 127.0.0.1:45572)
0806 22:39:58.243670 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.243716 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:58.243793 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:58.243958 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:58.243961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:58.244350 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:58.244434 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.244436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.245329 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.245330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.246235 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.246239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.246622 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.246623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.246717 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.247247 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:58.247256 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:58.248893 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:58.250546 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.250553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.250554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.250884 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.251083 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.251083 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.251084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.251141 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.251462 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:58.251471 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:58.251525 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 22:39:58.251641 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:39:58.251770 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
I20260806 22:39:58.253048 4120 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:39:58.260248 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.253350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:34723 (local address 127.0.0.1:55268)
0806 22:39:58.253400 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.253410 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:58.253452 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:58.253627 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:58.253628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:58.253780 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:58.253817 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.253819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.254516 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.254517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.255051 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.255053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.255445 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.255446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.255506 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.255999 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:58.256004 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:58.257277 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:58.259205 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.259205 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.259206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.259376 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.259587 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.259587 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.259587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.259630 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.259894 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:58.259895 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:58.259946 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:39:58.260021 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:39:58.260114 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:39:58.260272 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.253434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55268 (local address 127.16.235.125:34723)
0806 22:39:58.253487 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.253490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.253499 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:39:58.253527 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.253528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.253537 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.253606 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.253916 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.254436 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.255142 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.255367 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.255450 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.257382 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.257383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.257383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.257390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.259113 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.259452 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.259454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.259455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.259518 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.259666 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.259666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.259667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.259806 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.259862 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.260088 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.260181 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:39:58.260685 3887 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } attempt: 2
I20260806 22:39:58.260895 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.252777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45586 (local address 127.16.235.126:33433)
0806 22:39:58.252884 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.252886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.252892 (+ 6us) server_negotiation.cc:407] Connection header received
0806 22:39:58.252953 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.252954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.252982 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.253031 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.253641 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.254216 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.255157 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.255433 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.255476 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.258114 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.258115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.258116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.258123 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.259665 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.260045 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.260046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.260047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.260108 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.260328 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.260329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.260329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.260460 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.260662 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.260703 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.260802 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 22:39:58.260854 4120 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
I20260806 22:39:58.269958 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.261029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52718)
0806 22:39:58.261075 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.261087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:58.261126 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:58.261299 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:58.261300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:58.261474 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:58.261521 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.261523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.262465 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.262466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.263111 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.263113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.263596 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.263597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.263656 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.264108 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:58.264112 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:58.265707 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:58.268778 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.268779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.268780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.269001 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.269250 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.269251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.269251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.269318 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.269625 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:58.269626 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:58.269676 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:39:58.269723 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:39:58.269814 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 22:39:58.270350 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.261134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.16.235.124:44195)
0806 22:39:58.261175 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.261177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.261184 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:39:58.261204 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.261205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.261216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.261595 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.261627 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.262375 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.263205 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.263518 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.263597 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.265816 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.265817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.265817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.265825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.268674 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.269097 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.269098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.269098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.269156 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.269399 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.269399 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.269400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.269543 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.270094 (+ 551us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.270141 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.270223 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":22}
I20260806 22:39:58.271238 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.261092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52726)
0806 22:39:58.261216 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.261227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:58.261269 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:58.261760 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:58.261761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:58.261901 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:58.261927 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.261928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.263235 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.263236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.263967 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.263971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.264468 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.264469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.264524 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.265000 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:58.265006 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:58.266637 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:58.269873 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.269874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.269875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.270080 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.270267 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.270268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.270268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.270318 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.270912 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:58.270913 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:58.270955 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:39:58.271025 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:39:58.271132 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 22:39:58.271672 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.261159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.16.235.124:44195)
0806 22:39:58.261654 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.261655 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.261663 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:39:58.261682 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.261683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.261692 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.261759 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.262018 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.263158 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.264051 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.264387 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.264461 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.266791 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.266792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.266793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.266800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.269782 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.270154 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.270155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.270155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.270207 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.270388 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.270627 (+ 239us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.270628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.270830 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.270907 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.271071 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.271228 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":473,"thread_start_us":112,"threads_started":1}
I20260806 22:39:58.271894 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.261533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.16.235.125:34723)
0806 22:39:58.261572 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.261574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.261583 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:39:58.261605 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.261606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.261617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.261687 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.262239 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.263058 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.264327 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.264611 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.264685 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.267525 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.267526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.267527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.267534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.269607 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.270300 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.270301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.270302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.270398 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.270896 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.270897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.270897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.271050 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.271228 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.271689 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.271781 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:39:58.272324 3887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50: Bootstrap starting.
I20260806 22:39:58.273135 3887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:58.273454 3887 log.cc:824] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:58.273914 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:39:58.274732 4120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361: Bootstrap starting.
I20260806 22:39:58.275405 4120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:58.275730 4120 log.cc:824] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:58.276407 3887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50: No bootstrap required, opened a new log
I20260806 22:39:58.278513 3887 raft_consensus.cc:348] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } }
I20260806 22:39:58.278723 3887 raft_consensus.cc:374] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:58.278769 3887 raft_consensus.cc:729] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e12f416a52b841889e96699402a7db50, State: Initialized, Role: FOLLOWER
I20260806 22:39:58.278908 3887 consensus_queue.cc:260] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } }
I20260806 22:39:58.279292 3887 sys_catalog.cc:564] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:58.279238 4141 sys_catalog.cc:455] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.279678 4141 sys_catalog.cc:458] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:39:58.281232 4120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361: No bootstrap required, opened a new log
I20260806 22:39:58.283995 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.275738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.16.235.125:34723)
0806 22:39:58.275778 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.275780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.276130 (+ 350us) server_negotiation.cc:407] Connection header received
0806 22:39:58.276234 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.276235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.276246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.276278 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.276930 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.277387 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.278091 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.278365 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.278430 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.280107 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.280108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.280108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.280116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.282294 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.282760 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.282761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.282762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.282833 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.283048 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.283049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.283050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.283185 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.283275 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.283810 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.283895 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:39:58.285087 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.275425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.16.235.126:33433)
0806 22:39:58.275466 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.275469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.275578 (+ 109us) server_negotiation.cc:407] Connection header received
0806 22:39:58.275642 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.275643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.275657 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.275721 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.276553 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.277245 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.278269 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.278542 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.278620 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.280485 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.280486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.280486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.280494 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.283070 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.283466 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.283467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.283467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.283528 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.283734 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.283735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.283735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.284012 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.284477 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.284921 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.285001 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:39:58.284015 3783 master_runner.cc:186] Error getting master registration for 127.16.235.125:34723: OK, instance_id { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" instance_seqno: 1786055998075909 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:39:58.287145 3783 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:39:58.287704 4120 raft_consensus.cc:348] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } }
I20260806 22:39:58.287981 4120 raft_consensus.cc:374] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:58.288086 4120 raft_consensus.cc:729] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36e60bdb3c3438c86cff6917156c361, State: Initialized, Role: FOLLOWER
I20260806 22:39:58.288239 4120 consensus_queue.cc:260] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } }
I20260806 22:39:58.288616 4120 sys_catalog.cc:564] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:58.288594 4163 sys_catalog.cc:455] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.288684 4163 sys_catalog.cc:458] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:39:58.281514 4158 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:39:58.290388 4158 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 22:39:58.291006 4178 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:39:58.291060 4178 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:39:58.291221 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.275938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.16.235.124:44195)
0806 22:39:58.275983 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.275985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.283619 (+ 7634us) server_negotiation.cc:407] Connection header received
0806 22:39:58.283683 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.283684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.283711 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.283768 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.284341 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.285193 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.286132 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.286441 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.286519 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.287883 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.287885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.287885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.287892 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.289605 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.289984 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.289985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.289985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.290041 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.290261 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.290261 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.290262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.290417 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.290535 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.291048 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.291136 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:39:58.292642 4009 master_runner.cc:186] Error getting master registration for 127.16.235.125:34723: OK, instance_id { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" instance_seqno: 1786055998075909 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:39:58.295037 4009 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:39:58.314435 3993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } attempt: 3
Aug 06 22:39:58 dist-test-slave-3kk6 krb5kdc[3765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786055998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
I20260806 22:39:58.323174 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.314802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:44195 (local address 127.0.0.1:52756)
0806 22:39:58.314868 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:39:58.314880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:39:58.314921 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:39:58.315076 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:39:58.315077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:39:58.315292 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:39:58.315351 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.315353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.316324 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.316326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.317156 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:39:58.317160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.317656 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.317657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:39:58.317720 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.318309 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:39:58.318315 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:39:58.319997 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:39:58.322033 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.322034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.322035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.322251 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.322459 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:39:58.322460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:39:58.322460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:39:58.322516 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:39:58.322843 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:39:58.322844 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:39:58.322891 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:39:58.322955 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:39:58.323061 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:39:58.323560 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:39:58.314897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52756 (local address 127.16.235.124:44195)
0806 22:39:58.314940 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:39:58.314943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:39:58.314952 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:39:58.314977 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:39:58.314978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:39:58.314988 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:39:58.315056 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:39:58.315445 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.316218 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.317260 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:39:58.317571 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:39:58.317647 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:39:58.320112 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:39:58.320113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:39:58.320114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:39:58.320122 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:39:58.321929 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.322329 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.322330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.322331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.322389 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:39:58.322581 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:39:58.322581 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:39:58.322582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:39:58.322742 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:39:58.322805 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:39:58.323385 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 22:39:58.323469 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:39:58.325330 3993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e: Bootstrap starting.
I20260806 22:39:58.326149 3993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e: Neither blocks nor log segments found. Creating new log.
I20260806 22:39:58.326489 3993 log.cc:824] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:39:58.327518 3993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e: No bootstrap required, opened a new log
I20260806 22:39:58.329592 3993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } }
I20260806 22:39:58.329754 3993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:39:58.329792 3993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f14ad9fafa84403a88edf8390b3474e, State: Initialized, Role: FOLLOWER
I20260806 22:39:58.329913 3993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } }
I20260806 22:39:58.330255 3993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:39:58.330811 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.330973 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:39:58.332280 4201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:39:58.332391 4201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:39:58.334206 3897 master_runner.cc:263] No leader master found from master 1f14ad9fafa84403a88edf8390b3474e
I20260806 22:39:58.334265 3897 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:39:58.370770 4141 raft_consensus.cc:482] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:39:58.370946 4141 raft_consensus.cc:504] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } }
I20260806 22:39:58.371230 4141 leader_election.cc:288] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f14ad9fafa84403a88edf8390b3474e (127.16.235.125:34723), e36e60bdb3c3438c86cff6917156c361 (127.16.235.124:44195)
I20260806 22:39:58.371701 3960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e12f416a52b841889e96699402a7db50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f14ad9fafa84403a88edf8390b3474e" is_pre_election: true
I20260806 22:39:58.371914 3960 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e12f416a52b841889e96699402a7db50 in term 0.
I20260806 22:39:58.372153 3806 leader_election.cc:302] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f14ad9fafa84403a88edf8390b3474e, e12f416a52b841889e96699402a7db50; no voters:
I20260806 22:39:58.372318 4141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:39:58.372385 4141 raft_consensus.cc:482] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:39:58.372417 4141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:58.373502 4141 raft_consensus.cc:504] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } }
I20260806 22:39:58.373658 4141 leader_election.cc:288] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [CANDIDATE]: Term 1 election: Requested vote from peers 1f14ad9fafa84403a88edf8390b3474e (127.16.235.125:34723), e36e60bdb3c3438c86cff6917156c361 (127.16.235.124:44195)
I20260806 22:39:58.374156 4070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e12f416a52b841889e96699402a7db50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e60bdb3c3438c86cff6917156c361"
I20260806 22:39:58.374325 4070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:58.374759 4072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e12f416a52b841889e96699402a7db50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e36e60bdb3c3438c86cff6917156c361" is_pre_election: true
I20260806 22:39:58.375684 4070 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e12f416a52b841889e96699402a7db50 in term 1.
I20260806 22:39:58.375921 3960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e12f416a52b841889e96699402a7db50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f14ad9fafa84403a88edf8390b3474e"
I20260806 22:39:58.376022 3960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:39:58.377434 3960 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e12f416a52b841889e96699402a7db50 in term 1.
I20260806 22:39:58.377856 3808 leader_election.cc:302] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e12f416a52b841889e96699402a7db50, e36e60bdb3c3438c86cff6917156c361; no voters:
I20260806 22:39:58.378113 4141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:39:58.378316 4141 raft_consensus.cc:686] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [term 1 LEADER]: Becoming Leader. State: Replica: e12f416a52b841889e96699402a7db50, State: Running, Role: LEADER
I20260806 22:39:58.378428 4141 consensus_queue.cc:237] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } }
I20260806 22:39:58.378913 4141 sys_catalog.cc:455] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e12f416a52b841889e96699402a7db50. Latest consensus state: current_term: 1 leader_uuid: "e12f416a52b841889e96699402a7db50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.379050 4141 sys_catalog.cc:458] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:58.381204 4206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:39:58.381428 4206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:39:58.383364 4070 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [term 1 FOLLOWER]: Refusing update from remote peer e12f416a52b841889e96699402a7db50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:58.383664 3960 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [term 1 FOLLOWER]: Refusing update from remote peer e12f416a52b841889e96699402a7db50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:39:58.384224 4141 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:58.384394 4141 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:39:58.385851 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e12f416a52b841889e96699402a7db50. Latest consensus state: current_term: 1 leader_uuid: "e12f416a52b841889e96699402a7db50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.385958 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:39:58.387109 4163 sys_catalog.cc:455] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e12f416a52b841889e96699402a7db50. Latest consensus state: current_term: 1 leader_uuid: "e12f416a52b841889e96699402a7db50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.387239 4163 sys_catalog.cc:458] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:39:58.389098 4141 sys_catalog.cc:455] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e12f416a52b841889e96699402a7db50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.389196 4141 sys_catalog.cc:458] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:58.389679 4163 sys_catalog.cc:455] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e12f416a52b841889e96699402a7db50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.389784 4163 sys_catalog.cc:458] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:39:58.389866 4207 mvcc.cc:205] Tried to move back new op lower bound from 7315685369374691328 to 7315685369358708736. Current Snapshot: MvccSnapshot[applied={T|T < 7315685369374691328}]
I20260806 22:39:58.390058 4210 mvcc.cc:205] Tried to move back new op lower bound from 7315685369374691328 to 7315685369358708736. Current Snapshot: MvccSnapshot[applied={T|T < 7315685369374691328}]
I20260806 22:39:58.390456 4141 sys_catalog.cc:455] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e12f416a52b841889e96699402a7db50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.390547 4141 sys_catalog.cc:458] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50 [sys.catalog]: This master's current role is: LEADER
I20260806 22:39:58.390987 4181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e12f416a52b841889e96699402a7db50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12f416a52b841889e96699402a7db50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33433 } } peers { permanent_uuid: "1f14ad9fafa84403a88edf8390b3474e" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 34723 } } peers { permanent_uuid: "e36e60bdb3c3438c86cff6917156c361" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 44195 } } }
I20260806 22:39:58.391091 4181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:39:58.392968 4208 mvcc.cc:205] Tried to move back new op lower bound from 7315685369374691328 to 7315685369358708736. Current Snapshot: MvccSnapshot[applied={T|T < 7315685369374691328}]
I20260806 22:39:58.394093 4206 catalog_manager.cc:1347] Generated new cluster ID: 5286d6131e9146a3a148c02c7a32c647
I20260806 22:39:58.394162 4206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:39:58.407927 4206 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:39:58.408710 4206 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:39:58.419870 4206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e12f416a52b841889e96699402a7db50: Generated new TSK 0
I20260806 22:39:58.420106 4206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:39:59.291572 4178 catalog_manager.cc:1259] Loaded cluster ID: 5286d6131e9146a3a148c02c7a32c647
I20260806 22:39:59.291715 4178 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361: loading cluster ID for follower catalog manager: success
I20260806 22:39:59.292742 4178 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361: acquiring CA information for follower catalog manager: success
I20260806 22:39:59.293157 4178 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e36e60bdb3c3438c86cff6917156c361: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:39:59.332963 4201 catalog_manager.cc:1259] Loaded cluster ID: 5286d6131e9146a3a148c02c7a32c647
I20260806 22:39:59.333060 4201 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e: loading cluster ID for follower catalog manager: success
I20260806 22:39:59.333845 4201 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e: acquiring CA information for follower catalog manager: success
I20260806 22:39:59.334095 4201 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f14ad9fafa84403a88edf8390b3474e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:39:59.475857 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 3783
I20260806 22:39:59.486178 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 3897
I20260806 22:39:59.496121 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 4009
2026-08-06T22:39:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3085 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:39:59 dist-test-slave-3kk6 krb5kdc[4239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:39:59 dist-test-slave-3kk6 krb5kdc[4239](info): set up 2 sockets
Aug 06 22:39:59 dist-test-slave-3kk6 krb5kdc[4239](info): commencing operation
krb5kdc: starting...
W20260806 22:40:00.734133 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:40:00 dist-test-slave-3kk6 krb5kdc[4239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:40:00.870573 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40199
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:37261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:40199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:00.957610 4265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:00.957882 4265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:00.957926 4265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:00.959156 4265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:00.959233 4265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:00.959257 4265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:00.959280 4265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:00.959333 4265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:00.960865 4265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:37261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:40199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:00.961156 4265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:00.961408 4265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:00.966547 4271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:00.966596 4273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:00.966781 4265 server_base.cc:1034] running on GCE node
W20260806 22:40:00.966588 4275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:00.967149 4265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:00.967471 4265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:00.968649 4265 hybrid_clock.cc:648] HybridClock initialized: now 1786056000968643 us; error 40 us; skew 500 ppm
Aug 06 22:40:00 dist-test-slave-3kk6 krb5kdc[4239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:00.971406 4265 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:40:00.971865 4265 webserver.cc:460] Webserver started at http://127.16.235.126:37639/ using document root <none> and password file <none>
I20260806 22:40:00.972060 4265 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:00.972101 4265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:00.972198 4265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:00.973270 4265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "2688965c7a734b109a50b71386966ff3"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-3kk6"
server_key: "e181560e47d3cbf9f5c5d894008953ed"
server_key_iv: "c32b6fb4cde640904fda4453d4b9866b"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.973623 4265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "2688965c7a734b109a50b71386966ff3"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-3kk6"
server_key: "e181560e47d3cbf9f5c5d894008953ed"
server_key_iv: "c32b6fb4cde640904fda4453d4b9866b"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.975057 4265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:00.975818 4283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:00.976002 4265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:00.976091 4265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "2688965c7a734b109a50b71386966ff3"
format_stamp: "Formatted at 2026-08-06 22:40:00 on dist-test-slave-3kk6"
server_key: "e181560e47d3cbf9f5c5d894008953ed"
server_key_iv: "c32b6fb4cde640904fda4453d4b9866b"
server_key_version: "encryptionkey@0"
I20260806 22:40:00.976151 4265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:00.988941 4265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:00.991483 4265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:00.991613 4265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:00.997449 4265 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:40199
I20260806 22:40:00.997498 4359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:40199 every 8 connection(s)
I20260806 22:40:00.997992 4265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:00.999298 4360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:01.003127 4360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3: Bootstrap starting.
I20260806 22:40:01.003825 4360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:01.004101 4360 log.cc:824] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:01.004981 4360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3: No bootstrap required, opened a new log
I20260806 22:40:01.006608 4360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2688965c7a734b109a50b71386966ff3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40199 } }
I20260806 22:40:01.006899 4360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:01.006933 4360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2688965c7a734b109a50b71386966ff3, State: Initialized, Role: FOLLOWER
I20260806 22:40:01.007050 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 4265
I20260806 22:40:01.007107 4360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2688965c7a734b109a50b71386966ff3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40199 } }
I20260806 22:40:01.007182 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:40:01.007222 4360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:01.007266 4360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:01.007321 4360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:01.007452 17325 external_mini_cluster.cc:1428] Setting key cbab7c246df9e1d3dfeff2be2aa379c7
I20260806 22:40:01.008193 4360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2688965c7a734b109a50b71386966ff3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40199 } }
I20260806 22:40:01.008312 4360 leader_election.cc:302] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2688965c7a734b109a50b71386966ff3; no voters:
I20260806 22:40:01.008498 4360 leader_election.cc:288] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:01.008575 4367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:01.008777 4360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:01.008785 4367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [term 1 LEADER]: Becoming Leader. State: Replica: 2688965c7a734b109a50b71386966ff3, State: Running, Role: LEADER
I20260806 22:40:01.008920 4367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2688965c7a734b109a50b71386966ff3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40199 } }
I20260806 22:40:01.009334 4369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2688965c7a734b109a50b71386966ff3. Latest consensus state: current_term: 1 leader_uuid: "2688965c7a734b109a50b71386966ff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2688965c7a734b109a50b71386966ff3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40199 } } }
I20260806 22:40:01.009439 4369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:01.009603 4368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2688965c7a734b109a50b71386966ff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2688965c7a734b109a50b71386966ff3" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40199 } } }
I20260806 22:40:01.009737 4368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:01.010201 4383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:01.010877 4383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:01.013267 4383 catalog_manager.cc:1347] Generated new cluster ID: 5503a67e929f470a97339f045eff35a4
I20260806 22:40:01.013330 4383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:40:01 dist-test-slave-3kk6 krb5kdc[4239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:01.020556 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:01.008008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54354 (local address 127.16.235.126:40199)
0806 22:40:01.008279 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:01.008290 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:01.008333 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:40:01.008467 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:01.008475 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:01.008541 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:01.008700 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:01.009762 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.011327 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.012216 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:01.012822 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:01.012932 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:01.016159 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:01.016163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:01.016164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:01.016177 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:01.018572 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.019060 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.019064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.019065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.019242 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:01.019496 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:01.019497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:01.019497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:01.019856 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:01.020026 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:01.020133 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:40:01.020308 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":122,"threads_started":1}
I20260806 22:40:01.021695 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:40:01.031122 4383 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:01.031790 4383 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:01.038643 4383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2688965c7a734b109a50b71386966ff3: Generated new TSK 0
I20260806 22:40:01.038928 4383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:40:01.076295 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 4265
2026-08-06T22:40:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1580 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:01 dist-test-slave-3kk6 krb5kdc[4401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:01 dist-test-slave-3kk6 krb5kdc[4401](info): set up 2 sockets
Aug 06 22:40:01 dist-test-slave-3kk6 krb5kdc[4401](info): commencing operation
krb5kdc: starting...
W20260806 22:40:02.310825 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:40:02 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:40:02.449571 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45641
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:45641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:02.539532 4420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:02.539826 4420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:02.539871 4420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:02.541095 4420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:02.541178 4420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:02.541213 4420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:02.541235 4420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:02.541266 4420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:02.542917 4420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:45641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:02.543246 4420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:02.543514 4420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:02.548928 4430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:02.548913 4434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:02.548908 4432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:02.549139 4420 server_base.cc:1034] running on GCE node
I20260806 22:40:02.549465 4420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:02.549748 4420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:02.550942 4420 hybrid_clock.cc:648] HybridClock initialized: now 1786056002550920 us; error 43 us; skew 500 ppm
Aug 06 22:40:02 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:02.553910 4420 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:40:02.554435 4420 webserver.cc:460] Webserver started at http://127.16.235.126:43863/ using document root <none> and password file <none>
I20260806 22:40:02.554790 4420 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:02.554869 4420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:02.555042 4420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:02.556314 4420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "26812ef76a8c4a4483204e8b0b66d23a"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "c2b2edf158940773d6c4812f0fe0610f"
server_key_iv: "c39515ccac22d9d980d8f00ade81a003"
server_key_version: "encryptionkey@0"
I20260806 22:40:02.556720 4420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "26812ef76a8c4a4483204e8b0b66d23a"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "c2b2edf158940773d6c4812f0fe0610f"
server_key_iv: "c39515ccac22d9d980d8f00ade81a003"
server_key_version: "encryptionkey@0"
I20260806 22:40:02.558197 4420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:02.559149 4443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:02.559425 4420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:02.559548 4420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "26812ef76a8c4a4483204e8b0b66d23a"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "c2b2edf158940773d6c4812f0fe0610f"
server_key_iv: "c39515ccac22d9d980d8f00ade81a003"
server_key_version: "encryptionkey@0"
I20260806 22:40:02.559648 4420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:02.636066 4420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:02.638427 4420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:02.638622 4420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:02.643141 4420 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:45641
I20260806 22:40:02.643190 4521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:45641 every 8 connection(s)
I20260806 22:40:02.643606 4420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:02.644658 4522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:02.647195 4522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Bootstrap starting.
I20260806 22:40:02.647461 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 4420
I20260806 22:40:02.647581 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:40:02.647784 17325 external_mini_cluster.cc:1428] Setting key e898c7db72be2d59fceeab0525ca4b25
I20260806 22:40:02.648010 4522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:02.648393 4522 log.cc:824] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:02.649291 4522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: No bootstrap required, opened a new log
I20260806 22:40:02.651816 4522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:02.652006 4522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:02.652033 4522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26812ef76a8c4a4483204e8b0b66d23a, State: Initialized, Role: FOLLOWER
I20260806 22:40:02.652189 4522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:02.652280 4522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:02.652305 4522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:02.652376 4522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:02.653157 4522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:02.653275 4522 leader_election.cc:302] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26812ef76a8c4a4483204e8b0b66d23a; no voters:
I20260806 22:40:02.653475 4522 leader_election.cc:288] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:02.653529 4534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:02.653741 4534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 1 LEADER]: Becoming Leader. State: Replica: 26812ef76a8c4a4483204e8b0b66d23a, State: Running, Role: LEADER
I20260806 22:40:02.653817 4522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:02.653925 4534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:02.654290 4536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26812ef76a8c4a4483204e8b0b66d23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } } }
I20260806 22:40:02.654310 4538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26812ef76a8c4a4483204e8b0b66d23a. Latest consensus state: current_term: 1 leader_uuid: "26812ef76a8c4a4483204e8b0b66d23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } } }
I20260806 22:40:02.654490 4538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:02.654479 4536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:02.654974 4546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:02.655584 4546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:40:02 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:02.657564 4546 catalog_manager.cc:1347] Generated new cluster ID: 288ee965a3bc4557b44c18b92e40ee89
I20260806 22:40:02.657663 4546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:02.660629 4529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:02.648441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57582 (local address 127.16.235.126:45641)
0806 22:40:02.648677 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:02.648688 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:02.648724 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:40:02.649343 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:02.649353 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:02.649437 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:02.649596 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:02.650222 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:02.651723 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:02.652725 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:02.653399 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:02.653521 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:02.656828 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:02.656832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:02.656832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:02.656846 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:02.658973 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:02.659401 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:02.659405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:02.659405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:02.659503 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:02.659723 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:02.659725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:02.659725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:02.659979 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:02.660164 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:02.660256 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:40:02.660421 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":115,"threads_started":1}
I20260806 22:40:02.667596 4546 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:02.668345 4546 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:02.677744 4546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Generated new TSK 0
I20260806 22:40:02.678007 4546 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:40:02.709908 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45641
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:02.798727 4561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:02.798977 4561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:02.799005 4561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:02.800220 4561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:02.800278 4561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:02.800307 4561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:40:02.801977 4561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:02.802258 4561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:02.802536 4561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:02.808110 4568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:02.808116 4571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:02.808137 4561 server_base.cc:1034] running on GCE node
W20260806 22:40:02.808125 4569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:02.808601 4561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:02.808849 4561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:02.810007 4561 hybrid_clock.cc:648] HybridClock initialized: now 1786056002809975 us; error 56 us; skew 500 ppm
Aug 06 22:40:02 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:02.812935 4561 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:40:02.813429 4561 webserver.cc:460] Webserver started at http://127.16.235.65:40515/ using document root <none> and password file <none>
I20260806 22:40:02.813652 4561 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:02.813696 4561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:02.813795 4561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:02.815024 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "8cc88fd869954bc588eb232dcab876e1"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "ee4c2b00877051eba0e3eaa343afd427"
server_key_iv: "27da23bdf584539f58148627e1bca462"
server_key_version: "encryptionkey@0"
I20260806 22:40:02.815356 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "8cc88fd869954bc588eb232dcab876e1"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "ee4c2b00877051eba0e3eaa343afd427"
server_key_iv: "27da23bdf584539f58148627e1bca462"
server_key_version: "encryptionkey@0"
I20260806 22:40:02.816687 4561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:02.817456 4585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:02.817660 4561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:02.817757 4561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "8cc88fd869954bc588eb232dcab876e1"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "ee4c2b00877051eba0e3eaa343afd427"
server_key_iv: "27da23bdf584539f58148627e1bca462"
server_key_version: "encryptionkey@0"
I20260806 22:40:02.817823 4561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:02.842525 4561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:02.844774 4561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:02.844903 4561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:02.845129 4561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:02.845504 4561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:02.845542 4561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:02.845566 4561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:02.845620 4561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:02.851979 4561 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:43305
I20260806 22:40:02.852025 4755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:43305 every 8 connection(s)
I20260806 22:40:02.852396 4561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:02.856918 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 4561
I20260806 22:40:02.857023 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:02.857232 17325 external_mini_cluster.cc:1428] Setting key c466012aad5a7bc18ac9c0896985fe0d
Aug 06 22:40:02 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:02.863776 4760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:02.853125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:36083)
0806 22:40:02.853302 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:02.853350 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:02.853940 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:02.854291 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:02.854295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:02.855079 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:02.855292 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:02.855299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:02.856214 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:02.856215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:02.857102 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:02.857105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:02.857567 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:02.857568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:02.857725 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:02.858377 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:02.858389 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:02.860400 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:02.862088 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:02.862097 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:02.862099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:02.862407 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:02.862638 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:02.862639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:02.862639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:02.862801 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:02.863154 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:02.863163 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:02.863315 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 22:40:02.863435 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 22:40:02.863594 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260806 22:40:02.863757 4529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:02.853178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:36083 (local address 127.16.235.126:45641)
0806 22:40:02.853229 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:02.853232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:02.853447 (+ 215us) server_negotiation.cc:407] Connection header received
0806 22:40:02.854070 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:02.854071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:02.854083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:02.854188 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:02.855408 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:02.856114 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:02.857198 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:02.857470 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:02.857555 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:02.860541 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:02.860542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:02.860543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:02.860551 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:02.861979 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:02.862488 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:02.862489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:02.862490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:02.862559 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:02.862874 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:02.862875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:02.862876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:02.863053 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:02.863146 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:02.863513 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 22:40:02.863638 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:40:02.864429 4756 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45641
I20260806 22:40:02.864524 4756 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:02.864748 4756 heartbeater.cc:511] Master 127.16.235.126:45641 requested a full tablet report, sending...
I20260806 22:40:02.865396 4467 ts_manager.cc:194] Registered new tserver with Master: 8cc88fd869954bc588eb232dcab876e1 (127.16.235.65:43305)
I20260806 22:40:02.866393 4467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:36083
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:40:02.889717 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45641
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:02.979243 4763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:02.979477 4763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:02.979503 4763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:02.980649 4763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:02.980702 4763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:02.980751 4763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:40:02.982259 4763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:02.982477 4763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:02.982745 4763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:02.988085 4774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:02.988134 4763 server_base.cc:1034] running on GCE node
W20260806 22:40:02.988142 4771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:02.988106 4770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:02.988581 4763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:02.988946 4763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:02.990150 4763 hybrid_clock.cc:648] HybridClock initialized: now 1786056002990122 us; error 53 us; skew 500 ppm
Aug 06 22:40:02 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:02.997588 4763 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:40:02.998144 4763 webserver.cc:460] Webserver started at http://127.16.235.66:35345/ using document root <none> and password file <none>
I20260806 22:40:02.998426 4763 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:02.998502 4763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:02.998731 4763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:02.999956 4763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "47acc9cf0f004dae9bbaf7efa3c3440c"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "29420733466c14b146e35bda0d4dacbc"
server_key_iv: "b7b62ba8b4ac7807578efe1c71ad96b8"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.000378 4763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "47acc9cf0f004dae9bbaf7efa3c3440c"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "29420733466c14b146e35bda0d4dacbc"
server_key_iv: "b7b62ba8b4ac7807578efe1c71ad96b8"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.001932 4763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:03.002851 4786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.003057 4763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:03.003177 4763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "47acc9cf0f004dae9bbaf7efa3c3440c"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "29420733466c14b146e35bda0d4dacbc"
server_key_iv: "b7b62ba8b4ac7807578efe1c71ad96b8"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.003270 4763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.016048 4763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.018224 4763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.018441 4763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.018784 4763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:03.019188 4763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:03.019248 4763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.019295 4763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:03.019321 4763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.025902 4763 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:43145
I20260806 22:40:03.025961 4955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:43145 every 8 connection(s)
I20260806 22:40:03.026340 4763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:03.026603 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 4763
I20260806 22:40:03.026778 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:03.027052 17325 external_mini_cluster.cc:1428] Setting key 03682d196c463e9b6cc971f027678696
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056002, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
I20260806 22:40:03.038174 4529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.027401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:51953 (local address 127.16.235.126:45641)
0806 22:40:03.027454 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.027457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.027743 (+ 286us) server_negotiation.cc:407] Connection header received
0806 22:40:03.028373 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.028375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.028387 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.028435 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.029479 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.030287 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.031807 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.032018 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.032080 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.034893 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.034895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.034895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.034904 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.036555 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.037032 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.037034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.037034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.037099 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.037412 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.037412 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.037413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.037547 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.037621 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.037934 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.038074 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:03.038390 4962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.027376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:51953)
0806 22:40:03.027595 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.027647 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.028264 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.028522 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.028526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.029139 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.029358 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.029373 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.030414 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.030415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.031700 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.031705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.032116 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.032117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.032293 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.032917 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.032929 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.034774 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.036680 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.036690 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.036693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.036949 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.037170 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.037171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.037172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.037316 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.037626 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.037634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.037791 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.037910 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.038091 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260806 22:40:03.038990 4956 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45641
I20260806 22:40:03.039117 4956 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:03.039332 4956 heartbeater.cc:511] Master 127.16.235.126:45641 requested a full tablet report, sending...
I20260806 22:40:03.039754 4467 ts_manager.cc:194] Registered new tserver with Master: 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
I20260806 22:40:03.040225 4467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:51953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:40:03.058283 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45641
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:03.147571 4966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:03.147859 4966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:03.147905 4966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:03.149145 4966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:03.149221 4966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:03.149272 4966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:40:03.150939 4966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:03.151223 4966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:03.151481 4966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:03.157444 4976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.157457 4977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.157478 4980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.157768 4966 server_base.cc:1034] running on GCE node
I20260806 22:40:03.157994 4966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:03.158288 4966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:03.159471 4966 hybrid_clock.cc:648] HybridClock initialized: now 1786056003159455 us; error 36 us; skew 500 ppm
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:03.162449 4966 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:40:03.163094 4966 webserver.cc:460] Webserver started at http://127.16.235.67:43165/ using document root <none> and password file <none>
I20260806 22:40:03.163357 4966 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:03.163408 4966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:03.163556 4966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:03.164834 4966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-3kk6"
server_key: "388bacdcc6a6fe26e86af30b95fb8256"
server_key_iv: "6a24076157588f2fdc65880ba4937d24"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.165226 4966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-3kk6"
server_key: "388bacdcc6a6fe26e86af30b95fb8256"
server_key_iv: "6a24076157588f2fdc65880ba4937d24"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.166739 4966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:03.167644 4996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.167860 4966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:03.167972 4966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-3kk6"
server_key: "388bacdcc6a6fe26e86af30b95fb8256"
server_key_iv: "6a24076157588f2fdc65880ba4937d24"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.168069 4966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.186491 4966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.189266 4966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.189476 4966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.189766 4966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:03.190135 4966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:03.190196 4966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.190233 4966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:03.190282 4966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.196863 4966 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:33515
I20260806 22:40:03.196939 5154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:33515 every 8 connection(s)
I20260806 22:40:03.197340 4966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:03.205416 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 4966
I20260806 22:40:03.205544 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:03.205767 17325 external_mini_cluster.cc:1428] Setting key 12a186f6ec8cd40cc240d921bfd1a87c
I20260806 22:40:03.207796 4529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.198233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:40331 (local address 127.16.235.126:45641)
0806 22:40:03.198307 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.198309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.198565 (+ 256us) server_negotiation.cc:407] Connection header received
0806 22:40:03.199182 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.199183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.199196 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.199266 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.200352 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.200860 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.201924 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.202106 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.202159 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.204619 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.204620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.204620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.204628 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.206315 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.206914 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.206915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.206916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.206987 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.207238 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.207239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.207239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.207360 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.207416 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.207654 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.207727 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:40:03.208003 5157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.198235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:40331)
0806 22:40:03.198413 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.198460 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.199071 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.199351 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.199354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.200010 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.200233 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.200239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.200981 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.200983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.201839 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.201842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.202186 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.202187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.202295 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.202925 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.202935 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.204514 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.206408 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.206419 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.206422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.206824 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.207054 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.207054 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.207055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.207168 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.207405 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.207408 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.207535 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.207613 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.207749 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260806 22:40:03.208503 5155 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45641
I20260806 22:40:03.208612 5155 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:03.208774 5155 heartbeater.cc:511] Master 127.16.235.126:45641 requested a full tablet report, sending...
I20260806 22:40:03.209142 4467 ts_manager.cc:194] Registered new tserver with Master: c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515)
I20260806 22:40:03.209525 4467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:40331
I20260806 22:40:03.218011 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:40:03.226289 4529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.219632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57596 (local address 127.16.235.126:45641)
0806 22:40:03.219706 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.219708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.219758 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:40:03.219805 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.219806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.219816 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.219867 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.220554 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.221105 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.221892 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.222082 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.222143 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.223538 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.223540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.223540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.223547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.225105 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.225456 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.225457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.225458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.225516 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.225688 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.225688 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.225688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.225873 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.225960 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.226141 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.226214 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:40:03.228217 4467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:03.229883 4467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:03.237247 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.233079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.16.235.67:33515)
0806 22:40:03.233295 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.233301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.233313 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:40:03.233362 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.233364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.233378 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.233479 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.233673 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.234419 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.236048 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.236824 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.236909 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.236975 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.237134 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.237170 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":108,"threads_started":1}
I20260806 22:40:03.237252 5175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.232725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:33515 (local address 127.0.0.1:45406)
0806 22:40:03.232915 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.232960 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.233036 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.233488 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.233491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.233493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.233579 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.233582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.234537 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.234539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.235958 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.235960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.236921 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.236922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.237012 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.237028 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.237108 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.237159 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":41,"thread_start_us":62,"threads_started":1}
I20260806 22:40:03.237635 5174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.232731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:43145 (local address 127.0.0.1:47978)
0806 22:40:03.233102 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.233112 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.233151 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.233556 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.233557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.233559 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.233598 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.233600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.234612 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.234613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.236028 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.236031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.237415 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.237416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.237478 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.237481 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.237535 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.237573 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":59,"thread_start_us":119,"threads_started":1}
I20260806 22:40:03.237708 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.232851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47978 (local address 127.16.235.66:43145)
0806 22:40:03.233358 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.233363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.233377 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:40:03.233420 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.233422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.233438 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.233678 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.233760 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.234471 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.236116 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.236824 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.236909 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.236975 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.237559 (+ 584us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.237608 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":112,"threads_started":1}
I20260806 22:40:03.238129 5059 tablet_service.cc:1467] Processing CreateTablet for tablet 82e51af65ff846f1861ae731f9fc0154 (DEFAULT_TABLE table=test-table [id=1d87438745064159b9f3fd5260aa81ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:03.238574 5059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82e51af65ff846f1861ae731f9fc0154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.239099 5177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.232958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:43305 (local address 127.0.0.1:49796)
0806 22:40:03.234694 (+ 1736us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.234706 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.234766 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.235078 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.235079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.235081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.235120 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.235122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.236407 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.236408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.237268 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.237271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.238923 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.238924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.238971 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.238975 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.239024 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.239053 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1667,"thread_start_us":77,"threads_started":1}
I20260806 22:40:03.239176 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.233081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49796 (local address 127.16.235.65:43305)
0806 22:40:03.233301 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.233304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.234770 (+ 1466us) server_negotiation.cc:407] Connection header received
0806 22:40:03.234834 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.234837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.234855 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.235086 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.235193 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.236297 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.238001 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.238832 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.238912 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.238970 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.239039 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.239088 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":104,"threads_started":1}
I20260806 22:40:03.239811 4658 tablet_service.cc:1467] Processing CreateTablet for tablet 82e51af65ff846f1861ae731f9fc0154 (DEFAULT_TABLE table=test-table [id=1d87438745064159b9f3fd5260aa81ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:03.240140 4658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82e51af65ff846f1861ae731f9fc0154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.241717 5183 tablet_bootstrap.cc:492] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: Bootstrap starting.
I20260806 22:40:03.242465 5183 tablet_bootstrap.cc:654] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.242753 4858 tablet_service.cc:1467] Processing CreateTablet for tablet 82e51af65ff846f1861ae731f9fc0154 (DEFAULT_TABLE table=test-table [id=1d87438745064159b9f3fd5260aa81ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:03.243114 5183 log.cc:824] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:03.243163 4858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82e51af65ff846f1861ae731f9fc0154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:03.243168 5184 tablet_bootstrap.cc:492] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Bootstrap starting.
I20260806 22:40:03.243989 5183 tablet_bootstrap.cc:492] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: No bootstrap required, opened a new log
I20260806 22:40:03.244082 5184 tablet_bootstrap.cc:654] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.244081 5183 ts_tablet_manager.cc:1397] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:03.244689 5184 log.cc:824] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:03.245541 5184 tablet_bootstrap.cc:492] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: No bootstrap required, opened a new log
I20260806 22:40:03.245621 5184 ts_tablet_manager.cc:1397] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:40:03.246135 5188 tablet_bootstrap.cc:492] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: Bootstrap starting.
I20260806 22:40:03.246308 5183 raft_consensus.cc:348] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.246487 5183 raft_consensus.cc:374] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.246524 5183 raft_consensus.cc:729] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c584b6ec73e54f8c8fb4d969ae57ee6e, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.246641 5183 consensus_queue.cc:260] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.246888 5188 tablet_bootstrap.cc:654] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:03.247026 5183 ts_tablet_manager.cc:1428] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260806 22:40:03.247171 5155 heartbeater.cc:503] Master 127.16.235.126:45641 was elected leader, sending a full tablet report...
I20260806 22:40:03.247574 5188 log.cc:824] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:03.248214 5184 raft_consensus.cc:348] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.248417 5188 tablet_bootstrap.cc:492] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: No bootstrap required, opened a new log
I20260806 22:40:03.248409 5184 raft_consensus.cc:374] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.248464 5184 raft_consensus.cc:729] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc88fd869954bc588eb232dcab876e1, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.248492 5188 ts_tablet_manager.cc:1397] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:03.248598 5184 consensus_queue.cc:260] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.248893 5184 ts_tablet_manager.cc:1428] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:40:03.248924 4756 heartbeater.cc:503] Master 127.16.235.126:45641 was elected leader, sending a full tablet report...
I20260806 22:40:03.250538 5188 raft_consensus.cc:348] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.250734 5188 raft_consensus.cc:374] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:03.250775 5188 raft_consensus.cc:729] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47acc9cf0f004dae9bbaf7efa3c3440c, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.250869 5188 consensus_queue.cc:260] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.251106 5188 ts_tablet_manager.cc:1428] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:40:03.251144 4956 heartbeater.cc:503] Master 127.16.235.126:45641 was elected leader, sending a full tablet report...
W20260806 22:40:03.277227 4957 tablet.cc:2369] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:03.345250 5192 raft_consensus.cc:482] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:03.345423 5192 raft_consensus.cc:504] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.345889 5192 leader_election.cc:288] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515), 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145)
I20260806 22:40:03.349086 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.346173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:43145 (local address 127.16.235.65:54367)
0806 22:40:03.346398 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.346409 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.346481 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.346629 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.346629 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.346631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.346743 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.346745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.347393 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.347394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.348176 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.348178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.348776 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.348777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.348866 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.348870 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.348925 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.348961 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":70,"thread_start_us":96,"threads_started":1}
I20260806 22:40:03.349082 5176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.346237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:54367 (local address 127.16.235.66:43145)
0806 22:40:03.346286 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.346288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.346508 (+ 220us) server_negotiation.cc:407] Connection header received
0806 22:40:03.346545 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.346546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.346559 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.346628 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.346821 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.347273 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.348287 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.348680 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.348749 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.348757 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.348956 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.349007 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:40:03.349144 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.346222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:50863 (local address 127.16.235.67:33515)
0806 22:40:03.346278 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.346280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.346341 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:40:03.346442 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.346443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.346454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.346516 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:03.346737 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.347263 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.348318 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.348711 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.348812 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.348820 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.349016 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.349064 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:40:03.349098 4760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.346160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:33515 (local address 127.16.235.65:50863)
0806 22:40:03.346254 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.346270 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.346347 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.346542 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.346543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.346545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:03.346635 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.346637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.347367 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.347369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.348178 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.348180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.348831 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.348832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.348923 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.348928 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.348989 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.349030 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":2}
I20260806 22:40:03.349781 5089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82e51af65ff846f1861ae731f9fc0154" candidate_uuid: "8cc88fd869954bc588eb232dcab876e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" is_pre_election: true
I20260806 22:40:03.349769 4894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82e51af65ff846f1861ae731f9fc0154" candidate_uuid: "8cc88fd869954bc588eb232dcab876e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" is_pre_election: true
I20260806 22:40:03.349977 4894 raft_consensus.cc:2454] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cc88fd869954bc588eb232dcab876e1 in term 0.
I20260806 22:40:03.349974 5089 raft_consensus.cc:2454] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cc88fd869954bc588eb232dcab876e1 in term 0.
I20260806 22:40:03.350291 4588 leader_election.cc:302] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47acc9cf0f004dae9bbaf7efa3c3440c, 8cc88fd869954bc588eb232dcab876e1; no voters:
I20260806 22:40:03.350453 5192 raft_consensus.cc:2781] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:03.350518 5192 raft_consensus.cc:482] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:03.350559 5192 raft_consensus.cc:3037] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.351389 5192 raft_consensus.cc:504] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.351598 5192 leader_election.cc:288] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 1 election: Requested vote from peers c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515), 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145)
I20260806 22:40:03.351774 5089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82e51af65ff846f1861ae731f9fc0154" candidate_uuid: "8cc88fd869954bc588eb232dcab876e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e"
I20260806 22:40:03.351888 4894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82e51af65ff846f1861ae731f9fc0154" candidate_uuid: "8cc88fd869954bc588eb232dcab876e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c"
I20260806 22:40:03.351897 5089 raft_consensus.cc:3037] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.351984 4894 raft_consensus.cc:3037] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:03.352824 4894 raft_consensus.cc:2454] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc88fd869954bc588eb232dcab876e1 in term 1.
I20260806 22:40:03.353036 5089 raft_consensus.cc:2454] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cc88fd869954bc588eb232dcab876e1 in term 1.
I20260806 22:40:03.353132 4588 leader_election.cc:302] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47acc9cf0f004dae9bbaf7efa3c3440c, 8cc88fd869954bc588eb232dcab876e1; no voters:
W20260806 22:40:03.353181 4759 tablet.cc:2369] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:03.353243 5192 raft_consensus.cc:2781] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:03.353438 5192 raft_consensus.cc:686] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 1 LEADER]: Becoming Leader. State: Replica: 8cc88fd869954bc588eb232dcab876e1, State: Running, Role: LEADER
I20260806 22:40:03.353551 5192 consensus_queue.cc:237] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.354595 4467 catalog_manager.cc:5495] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cc88fd869954bc588eb232dcab876e1 (127.16.235.65). New cstate: current_term: 1 leader_uuid: "8cc88fd869954bc588eb232dcab876e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } health_report { overall_health: HEALTHY } } }
I20260806 22:40:03.360430 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.357817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49812 (local address 127.16.235.65:43305)
0806 22:40:03.357890 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.357892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.357899 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:40:03.357943 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.357943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.357963 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.358048 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:03.358207 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.358767 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.359556 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.359774 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.359856 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.360041 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:03.360158 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:03.360260 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.360283 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.360306 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:40:03.362571 5089 raft_consensus.cc:1261] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 1 FOLLOWER]: Refusing update from remote peer 8cc88fd869954bc588eb232dcab876e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:03.362571 4894 raft_consensus.cc:1261] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 1 FOLLOWER]: Refusing update from remote peer 8cc88fd869954bc588eb232dcab876e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:03.363067 5192 consensus_queue.cc:1035] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:03.363261 5195 consensus_queue.cc:1035] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:03.366261 5202 mvcc.cc:205] Tried to move back new op lower bound from 7315685389769744384 to 7315685389736980480. Current Snapshot: MvccSnapshot[applied={T|T < 7315685389769744384}]
I20260806 22:40:03.366531 5200 mvcc.cc:205] Tried to move back new op lower bound from 7315685389769744384 to 7315685389736980480. Current Snapshot: MvccSnapshot[applied={T|T < 7315685389769744384}]
I20260806 22:40:03.366685 5201 mvcc.cc:205] Tried to move back new op lower bound from 7315685389769744384 to 7315685389736980480. Current Snapshot: MvccSnapshot[applied={T|T < 7315685389769744384}]
I20260806 22:40:03.368980 4466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:03.369112 4466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:03.370014 4466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Sending DeleteTablet for 3 replicas of tablet 82e51af65ff846f1861ae731f9fc0154
I20260806 22:40:03.370414 5059 tablet_service.cc:1514] Processing DeleteTablet for tablet 82e51af65ff846f1861ae731f9fc0154 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:03 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:45406
I20260806 22:40:03.370433 4658 tablet_service.cc:1514] Processing DeleteTablet for tablet 82e51af65ff846f1861ae731f9fc0154 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:03 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:49796
I20260806 22:40:03.370606 4858 tablet_service.cc:1514] Processing DeleteTablet for tablet 82e51af65ff846f1861ae731f9fc0154 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:03 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47978
I20260806 22:40:03.370786 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 4561
I20260806 22:40:03.378142 5212 tablet_replica.cc:330] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: stopping tablet replica
I20260806 22:40:03.378404 5212 raft_consensus.cc:2229] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:03.378738 5212 raft_consensus.cc:2258] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:03.379464 5212 ts_tablet_manager.cc:1905] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:03.381758 5209 tablet_replica.cc:330] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: stopping tablet replica
I20260806 22:40:03.381943 5209 raft_consensus.cc:2229] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:03.381951 5212 ts_tablet_manager.cc:1918] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:03.382036 5212 log.cc:1196] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/82e51af65ff846f1861ae731f9fc0154
I20260806 22:40:03.382103 5209 raft_consensus.cc:2258] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:03.382419 5212 ts_tablet_manager.cc:1939] T 82e51af65ff846f1861ae731f9fc0154 P 47acc9cf0f004dae9bbaf7efa3c3440c: Deleting consensus metadata
I20260806 22:40:03.382849 5209 ts_tablet_manager.cc:1905] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:03.383049 4445 catalog_manager.cc:4841] TS 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145): tablet 82e51af65ff846f1861ae731f9fc0154 (table test-table [id=1d87438745064159b9f3fd5260aa81ff]) successfully deleted
I20260806 22:40:03.384984 5209 ts_tablet_manager.cc:1918] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:03.385085 5209 log.cc:1196] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/82e51af65ff846f1861ae731f9fc0154
I20260806 22:40:03.385419 5209 ts_tablet_manager.cc:1939] T 82e51af65ff846f1861ae731f9fc0154 P c584b6ec73e54f8c8fb4d969ae57ee6e: Deleting consensus metadata
I20260806 22:40:03.385998 4446 catalog_manager.cc:4841] TS c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515): tablet 82e51af65ff846f1861ae731f9fc0154 (table test-table [id=1d87438745064159b9f3fd5260aa81ff]) successfully deleted
W20260806 22:40:03.388370 4446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:43305 (error 108)
I20260806 22:40:03.388738 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 4763
I20260806 22:40:03.393860 5177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.388927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:43305 (local address 127.0.0.1:49816)
0806 22:40:03.393702 (+ 4775us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.393800 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4735}
W20260806 22:40:03.402469 4446 catalog_manager.cc:4584] TS 8cc88fd869954bc588eb232dcab876e1 (127.16.235.65:43305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 82e51af65ff846f1861ae731f9fc0154: Network error: Client connection negotiation failed: client connection to 127.16.235.65:43305: connect: Connection refused (error 111)
I20260806 22:40:03.412856 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 4966
I20260806 22:40:03.428499 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 4420
I20260806 22:40:03.443907 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45641
--webserver_interface=127.16.235.126
--webserver_port=43863
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:45641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:03.530627 5214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:03.531111 5214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:03.531222 5214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:03.532478 5214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:03.532554 5214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:03.532579 5214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:03.532601 5214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:03.532631 5214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:03.534163 5214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:45641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=43863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:03.534461 5214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:03.534741 5214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:03.544606 5228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.544561 5227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.544590 5232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.544871 5214 server_base.cc:1034] running on GCE node
I20260806 22:40:03.545147 5214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:03.545512 5214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:03.546764 5214 hybrid_clock.cc:648] HybridClock initialized: now 1786056003546749 us; error 50 us; skew 500 ppm
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:03.549664 5214 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:40:03.550206 5214 webserver.cc:460] Webserver started at http://127.16.235.126:43863/ using document root <none> and password file <none>
I20260806 22:40:03.550477 5214 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:03.550582 5214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:03.552155 5214 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:03.552970 5240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.553174 5214 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:03.553301 5214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "26812ef76a8c4a4483204e8b0b66d23a"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "c2b2edf158940773d6c4812f0fe0610f"
server_key_iv: "c39515ccac22d9d980d8f00ade81a003"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.553784 5214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.564675 5214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.566927 5214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.567111 5214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.571731 5214 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:45641
I20260806 22:40:03.571767 5320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:45641 every 8 connection(s)
I20260806 22:40:03.572230 5214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:03.574061 5321 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:40:03.574896 5321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Bootstrap starting.
I20260806 22:40:03.580147 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 5214
I20260806 22:40:03.580502 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:43305
--local_ip_for_outbound_sockets=127.16.235.65
--tserver_master_addrs=127.16.235.126:45641
--webserver_port=40515
--webserver_interface=127.16.235.65
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:03.584470 5321 log.cc:824] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:03.586621 5321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:40:03.587049 5321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Bootstrap complete.
I20260806 22:40:03.589510 5321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:03.589787 5321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26812ef76a8c4a4483204e8b0b66d23a, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.589984 5321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:03.590068 5321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:03.590134 5321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:03.590190 5321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:40:03.591415 5321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:03.591547 5321 leader_election.cc:302] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26812ef76a8c4a4483204e8b0b66d23a; no voters:
I20260806 22:40:03.591801 5321 leader_election.cc:288] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:40:03.591840 5330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:40:03.592012 5330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 2 LEADER]: Becoming Leader. State: Replica: 26812ef76a8c4a4483204e8b0b66d23a, State: Running, Role: LEADER
I20260806 22:40:03.592162 5321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:03.592170 5330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:03.592568 5330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26812ef76a8c4a4483204e8b0b66d23a. Latest consensus state: current_term: 2 leader_uuid: "26812ef76a8c4a4483204e8b0b66d23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } } }
I20260806 22:40:03.592731 5333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "26812ef76a8c4a4483204e8b0b66d23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } } }
I20260806 22:40:03.592828 5330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:03.592937 5333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:03.593305 5340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:03.594301 5340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:03.594509 5340 catalog_manager.cc:1259] Loaded cluster ID: 288ee965a3bc4557b44c18b92e40ee89
I20260806 22:40:03.594739 5340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:03.595059 5340 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 22:40:03.599804 5340 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 22:40:03.602814 5340 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:03.603068 5340 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Loaded TSK: 0
I20260806 22:40:03.603299 5340 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:40:03.672422 5328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:03.672660 5328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:03.672688 5328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:03.673966 5328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:03.674029 5328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:03.674060 5328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:40:03.675768 5328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:43305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=40515
--tserver_master_addrs=127.16.235.126:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:03.676083 5328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:03.676385 5328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:03.682054 5363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.681999 5371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.681986 5362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.682325 5328 server_base.cc:1034] running on GCE node
I20260806 22:40:03.682626 5328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:03.683008 5328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:03.684198 5328 hybrid_clock.cc:648] HybridClock initialized: now 1786056003684179 us; error 40 us; skew 500 ppm
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:03.687392 5328 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:40:03.688030 5328 webserver.cc:460] Webserver started at http://127.16.235.65:40515/ using document root <none> and password file <none>
I20260806 22:40:03.688350 5328 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:03.688444 5328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:03.690429 5328 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:40:03.691358 5380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.691558 5328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:03.691690 5328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "8cc88fd869954bc588eb232dcab876e1"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "ee4c2b00877051eba0e3eaa343afd427"
server_key_iv: "27da23bdf584539f58148627e1bca462"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.692102 5328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.718223 5328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.720846 5328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.721024 5328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.721315 5328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:03.721818 5389 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:40:03.723834 5328 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:40:03.723917 5328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:40:03.723959 5328 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:40:03.724690 5328 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:40:03.724754 5328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:03.724809 5389 tablet_bootstrap.cc:492] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Bootstrap starting.
I20260806 22:40:03.732707 5328 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:43305
I20260806 22:40:03.732800 5547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:43305 every 8 connection(s)
I20260806 22:40:03.733191 5328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:03.736642 5389 log.cc:824] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:03.737690 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 5328
I20260806 22:40:03.737999 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:43145
--local_ip_for_outbound_sockets=127.16.235.66
--tserver_master_addrs=127.16.235.126:45641
--webserver_port=35345
--webserver_interface=127.16.235.66
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:03.739506 5389 tablet_bootstrap.cc:492] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:40:03.739892 5389 tablet_bootstrap.cc:492] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Bootstrap complete.
I20260806 22:40:03.741674 5389 ts_tablet_manager.cc:1397] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.014s
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:03.743470 5389 raft_consensus.cc:348] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.743885 5389 raft_consensus.cc:729] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc88fd869954bc588eb232dcab876e1, State: Initialized, Role: FOLLOWER
I20260806 22:40:03.744065 5389 consensus_queue.cc:260] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:03.744480 5389 ts_tablet_manager.cc:1428] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:40:03.746938 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.733983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:47915)
0806 22:40:03.734219 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.734276 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.735183 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.735709 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.735714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.736197 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.736505 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.736514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.737882 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.737883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.739181 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.739184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.740149 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.740151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.740292 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.740984 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.740993 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.742622 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.745114 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.745122 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.745124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.745370 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.745678 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.745679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.745679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.745792 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.746219 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.746226 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.746370 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.746480 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.746718 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":117,"threads_started":1}
I20260806 22:40:03.746938 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.734173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:47915 (local address 127.16.235.126:45641)
0806 22:40:03.734341 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.734350 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.734396 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:40:03.735438 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.735444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.735493 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.735600 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.736639 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.737704 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.739297 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.740015 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.740164 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.742821 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.742825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.742825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.742837 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.744991 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.745483 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.745486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.745486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.745596 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.745869 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.745870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.745870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.746131 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.746228 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.746554 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.746721 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260806 22:40:03.747748 5548 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45641
I20260806 22:40:03.747876 5548 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:03.748103 5548 heartbeater.cc:511] Master 127.16.235.126:45641 requested a full tablet report, sending...
I20260806 22:40:03.748957 5264 ts_manager.cc:194] Registered new tserver with Master: 8cc88fd869954bc588eb232dcab876e1 (127.16.235.65:43305)
I20260806 22:40:03.750012 5264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:47915
I20260806 22:40:03.751021 5548 heartbeater.cc:503] Master 127.16.235.126:45641 was elected leader, sending a full tablet report...
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.65@KRBTEST.COM
I20260806 22:40:03.759624 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.749657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:43305 (local address 127.0.0.1:49822)
0806 22:40:03.749938 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.749979 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.750057 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.750482 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.750491 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.750947 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.751030 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.751034 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.752490 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.752491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.753073 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.753075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.753726 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.753727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.753784 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.754375 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.754382 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.756120 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.758357 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.758362 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.758364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.758581 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.758829 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.758829 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.758830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.758904 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.759307 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.759313 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.759362 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.759432 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.759547 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":124,"threads_started":1}
I20260806 22:40:03.759711 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.749726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49822 (local address 127.16.235.65:43305)
0806 22:40:03.750207 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.750212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.750223 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:40:03.750276 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.750278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.750295 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.750376 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.751142 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.752385 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.753161 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.753642 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.753720 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.756220 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.756231 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.756231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.756240 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.758269 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.758683 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.758686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.758687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.758760 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.758977 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.758978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.758979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.759212 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.759323 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.759497 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.759606 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":101,"threads_started":1}
I20260806 22:40:03.760134 5445 tablet_service.cc:1514] Processing DeleteTablet for tablet 82e51af65ff846f1861ae731f9fc0154 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:03 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:49822
I20260806 22:40:03.760764 5562 tablet_replica.cc:330] stopping tablet replica
I20260806 22:40:03.760887 5562 raft_consensus.cc:2229] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:03.760969 5562 raft_consensus.cc:2258] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:03.761303 5562 ts_tablet_manager.cc:1905] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:03.763433 5562 ts_tablet_manager.cc:1918] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:03.763512 5562 log.cc:1196] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/82e51af65ff846f1861ae731f9fc0154
I20260806 22:40:03.763768 5562 ts_tablet_manager.cc:1939] T 82e51af65ff846f1861ae731f9fc0154 P 8cc88fd869954bc588eb232dcab876e1: Deleting consensus metadata
I20260806 22:40:03.764376 5244 catalog_manager.cc:4841] TS 8cc88fd869954bc588eb232dcab876e1 (127.16.235.65:43305): tablet 82e51af65ff846f1861ae731f9fc0154 (table test-table [id=1d87438745064159b9f3fd5260aa81ff]) successfully deleted
W20260806 22:40:03.832732 5555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:03.833039 5555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:03.833086 5555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:03.834342 5555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:03.834431 5555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:03.834477 5555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:40:03.836115 5555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:43145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=35345
--tserver_master_addrs=127.16.235.126:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:03.836429 5555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:03.836697 5555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:40:03.842526 5555 server_base.cc:1034] running on GCE node
W20260806 22:40:03.842446 5576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.842445 5578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.842445 5571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.843004 5555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:03.843258 5555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:03.844409 5555 hybrid_clock.cc:648] HybridClock initialized: now 1786056003844400 us; error 35 us; skew 500 ppm
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:03.847390 5555 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:40:03.847922 5555 webserver.cc:460] Webserver started at http://127.16.235.66:35345/ using document root <none> and password file <none>
I20260806 22:40:03.848150 5555 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:03.848212 5555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:03.849598 5555 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:40:03.850278 5585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.850495 5555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:03.850595 5555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "47acc9cf0f004dae9bbaf7efa3c3440c"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "29420733466c14b146e35bda0d4dacbc"
server_key_iv: "b7b62ba8b4ac7807578efe1c71ad96b8"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.851071 5555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.862030 5555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.864722 5555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.864857 5555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.865074 5555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:03.865432 5555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:03.865473 5555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.865520 5555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:03.865547 5555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.872053 5555 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:43145
I20260806 22:40:03.872097 5752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:43145 every 8 connection(s)
I20260806 22:40:03.872519 5555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:03.876741 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 5555
I20260806 22:40:03.877053 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:33515
--local_ip_for_outbound_sockets=127.16.235.67
--tserver_master_addrs=127.16.235.126:45641
--webserver_port=43165
--webserver_interface=127.16.235.67
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:03.884114 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.873396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:38159 (local address 127.16.235.126:45641)
0806 22:40:03.873466 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:03.873469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:03.873855 (+ 386us) server_negotiation.cc:407] Connection header received
0806 22:40:03.874389 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:03.874390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:03.874404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:03.874460 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:03.875567 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.876083 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.877474 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.877681 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.877740 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.880646 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:03.880648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:03.880649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:03.880657 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:03.882430 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.882912 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.882914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.882915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.882999 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:03.883292 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:03.883293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:03.883294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:03.883469 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:03.883547 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:03.883846 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 22:40:03.883970 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:40:03.884126 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:03.873396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:38159)
0806 22:40:03.873698 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:03.873741 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:03.874301 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:03.874535 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:03.874538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:03.875263 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:03.875469 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.875474 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.876172 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.876173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.877382 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:03.877385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:03.877774 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:03.877775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:03.877956 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:03.878577 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:03.878589 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:03.880527 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:03.882520 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.882528 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.882530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.882829 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.883093 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:03.883094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:03.883094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:03.883209 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:03.883544 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:03.883550 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:03.883691 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 22:40:03.883779 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:40:03.883944 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":96,"threads_started":1}
I20260806 22:40:03.884753 5754 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45641
I20260806 22:40:03.884887 5754 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:03.885129 5754 heartbeater.cc:511] Master 127.16.235.126:45641 requested a full tablet report, sending...
I20260806 22:40:03.885581 5264 ts_manager.cc:194] Registered new tserver with Master: 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145)
I20260806 22:40:03.885973 5264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:38159
W20260806 22:40:03.966720 5757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:03.966974 5757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:03.967000 5757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:03.968206 5757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:03.968266 5757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:03.968295 5757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:40:03.969873 5757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:33515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=43165
--tserver_master_addrs=127.16.235.126:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:03.970178 5757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:03.970464 5757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:03.976066 5763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.976087 5768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:03.976166 5766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:03.976171 5757 server_base.cc:1034] running on GCE node
I20260806 22:40:03.976755 5757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:03.977119 5757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:03.978312 5757 hybrid_clock.cc:648] HybridClock initialized: now 1786056003978288 us; error 49 us; skew 500 ppm
Aug 06 22:40:03 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:03.981150 5757 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:40:03.981594 5757 webserver.cc:460] Webserver started at http://127.16.235.67:43165/ using document root <none> and password file <none>
I20260806 22:40:03.981858 5757 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:03.981945 5757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:03.983435 5757 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:03.984077 5779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.984302 5757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:03.984421 5757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-3kk6"
server_key: "388bacdcc6a6fe26e86af30b95fb8256"
server_key_iv: "6a24076157588f2fdc65880ba4937d24"
server_key_version: "encryptionkey@0"
I20260806 22:40:03.984783 5757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:03.996495 5757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:03.998610 5757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:03.998852 5757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:03.999125 5757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:03.999481 5757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:03.999516 5757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:03.999540 5757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:03.999588 5757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:04.005790 5757 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:33515
I20260806 22:40:04.005883 5943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:33515 every 8 connection(s)
I20260806 22:40:04.006249 5757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:40:04 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056003, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:04.014418 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 5757
I20260806 22:40:04.016894 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.007253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:57839 (local address 127.16.235.126:45641)
0806 22:40:04.007322 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.007324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.007539 (+ 215us) server_negotiation.cc:407] Connection header received
0806 22:40:04.008038 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.008040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.008049 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.008116 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:04.009109 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.009605 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.010785 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.010997 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.011062 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.013458 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:04.013459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:04.013459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:04.013467 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:04.015258 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:04.015820 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:04.015821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:04.015822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:04.015894 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:04.016202 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:04.016203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:04.016203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:04.016341 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:04.016415 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.016685 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.016777 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:40:04.017019 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.007200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:57839)
0806 22:40:04.007392 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.007440 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:04.007948 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:04.008171 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:04.008174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:04.008795 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:04.009008 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.009013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.009744 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.009745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.010625 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.010628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.011095 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.011096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.011217 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.011796 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:04.011806 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:04.013348 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:04.015353 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:04.015364 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:04.015367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:04.015734 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:04.015967 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:04.015968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:04.015968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:04.016134 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:04.016408 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:04.016413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:04.016549 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 22:40:04.016637 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:40:04.016776 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260806 22:40:04.017602 5944 heartbeater.cc:347] Connected to a master server at 127.16.235.126:45641
I20260806 22:40:04.017737 5944 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:04.017938 5944 heartbeater.cc:511] Master 127.16.235.126:45641 requested a full tablet report, sending...
I20260806 22:40:04.018445 5264 ts_manager.cc:194] Registered new tserver with Master: c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515)
I20260806 22:40:04.018900 5264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:57839
I20260806 22:40:04.022306 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.014957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57606 (local address 127.16.235.126:45641)
0806 22:40:04.015132 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.015134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.015199 (+ 65us) server_negotiation.cc:407] Connection header received
0806 22:40:04.015260 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.015261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.015274 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.015347 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:04.016202 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.016954 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.018138 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.018479 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.018555 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.019985 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:04.019986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:04.019986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:04.019993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:04.021299 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:04.021640 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:04.021640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:04.021641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:04.021695 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:04.021873 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:04.021874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:04.021874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:04.021987 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:04.022109 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.022165 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.022237 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":149,"thread_start_us":118,"threads_started":1}
I20260806 22:40:04.023226 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:40:04.026376 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.023657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57616 (local address 127.16.235.126:45641)
0806 22:40:04.023736 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.023738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.023860 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:40:04.023942 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.023942 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.023992 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.024082 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:04.024217 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.024719 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.025521 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.025750 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.025832 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.026016 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:04.026137 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:04.026239 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.026264 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.026290 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:40:04.026787 5264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:04.028473 5264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:04.031246 5445 tablet_service.cc:1467] Processing CreateTablet for tablet 6c25090e86d04fdfb6066ccd5141df54 (DEFAULT_TABLE table=test-table [id=d9f00c5df7fd4f4fa5344eea4926bba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:04.031502 5445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c25090e86d04fdfb6066ccd5141df54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:04.032785 5389 tablet_bootstrap.cc:492] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Bootstrap starting.
I20260806 22:40:04.033207 5389 tablet_bootstrap.cc:654] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:04.034828 5389 tablet_bootstrap.cc:492] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: No bootstrap required, opened a new log
I20260806 22:40:04.034842 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.030759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:43145 (local address 127.0.0.1:47980)
0806 22:40:04.031112 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.031123 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:04.031191 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:04.031519 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:04.031520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:04.031521 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:04.031558 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.031559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.032830 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.032831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.033800 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.033803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.034534 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.034535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.034598 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.034622 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 22:40:04.034717 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:40:04.034761 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":164,"threads_started":1}
I20260806 22:40:04.034897 5389 ts_tablet_manager.cc:1397] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:40:04.035077 5389 raft_consensus.cc:348] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.034855 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.030852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47980 (local address 127.16.235.66:43145)
0806 22:40:04.031050 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.031056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.031207 (+ 151us) server_negotiation.cc:407] Connection header received
0806 22:40:04.031265 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.031267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.031286 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.031375 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:04.031703 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.032745 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.033917 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.034441 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.034532 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.034598 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.034737 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.034778 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":82,"threads_started":1}
I20260806 22:40:04.035166 5389 raft_consensus.cc:374] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:04.035231 5389 raft_consensus.cc:729] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc88fd869954bc588eb232dcab876e1, State: Initialized, Role: FOLLOWER
I20260806 22:40:04.035296 5389 consensus_queue.cc:260] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.035416 5389 ts_tablet_manager.cc:1428] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:04.036010 5659 tablet_service.cc:1467] Processing CreateTablet for tablet 6c25090e86d04fdfb6066ccd5141df54 (DEFAULT_TABLE table=test-table [id=d9f00c5df7fd4f4fa5344eea4926bba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:04.036155 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.030743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:33515 (local address 127.0.0.1:45416)
0806 22:40:04.030861 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.030873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:04.030924 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:04.031257 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:04.031257 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:04.031259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:04.031317 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.031318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.033729 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.033730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.034512 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.034515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.035899 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.035900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.035974 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.035979 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:04.036039 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:40:04.036083 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 22:40:04.036173 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.030840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.16.235.67:33515)
0806 22:40:04.031052 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.031056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.031068 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:40:04.031116 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.031119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.031141 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.031409 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:04.031471 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.032649 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.034627 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.035786 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.035890 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.035949 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.036054 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.036093 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":125,"threads_started":1}
I20260806 22:40:04.036402 5659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c25090e86d04fdfb6066ccd5141df54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:04.036867 5848 tablet_service.cc:1467] Processing CreateTablet for tablet 6c25090e86d04fdfb6066ccd5141df54 (DEFAULT_TABLE table=test-table [id=d9f00c5df7fd4f4fa5344eea4926bba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:04.037213 5848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c25090e86d04fdfb6066ccd5141df54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:04.040316 5964 tablet_bootstrap.cc:492] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: Bootstrap starting.
I20260806 22:40:04.041039 5965 tablet_bootstrap.cc:492] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: Bootstrap starting.
I20260806 22:40:04.041172 5964 tablet_bootstrap.cc:654] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:04.041612 5965 tablet_bootstrap.cc:654] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:04.041742 5964 log.cc:824] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:04.042153 5965 log.cc:824] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:04.042706 5964 tablet_bootstrap.cc:492] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: No bootstrap required, opened a new log
I20260806 22:40:04.042793 5964 ts_tablet_manager.cc:1397] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:40:04.043046 5965 tablet_bootstrap.cc:492] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: No bootstrap required, opened a new log
I20260806 22:40:04.043119 5965 ts_tablet_manager.cc:1397] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:40:04.044982 5964 raft_consensus.cc:348] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.045204 5964 raft_consensus.cc:374] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:04.045245 5964 raft_consensus.cc:729] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47acc9cf0f004dae9bbaf7efa3c3440c, State: Initialized, Role: FOLLOWER
I20260806 22:40:04.045320 5965 raft_consensus.cc:348] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.045392 5964 consensus_queue.cc:260] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.045476 5965 raft_consensus.cc:374] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:04.045514 5965 raft_consensus.cc:729] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c584b6ec73e54f8c8fb4d969ae57ee6e, State: Initialized, Role: FOLLOWER
I20260806 22:40:04.045639 5965 consensus_queue.cc:260] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.045738 5964 ts_tablet_manager.cc:1428] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:40:04.045791 5754 heartbeater.cc:503] Master 127.16.235.126:45641 was elected leader, sending a full tablet report...
I20260806 22:40:04.045930 5965 ts_tablet_manager.cc:1428] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:40:04.046034 5944 heartbeater.cc:503] Master 127.16.235.126:45641 was elected leader, sending a full tablet report...
W20260806 22:40:04.123178 5755 tablet.cc:2369] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:04.191136 5970 raft_consensus.cc:482] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:04.191342 5970 raft_consensus.cc:504] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.191835 5970 leader_election.cc:288] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515), 8cc88fd869954bc588eb232dcab876e1 (127.16.235.65:43305)
I20260806 22:40:04.195948 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.192177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:56739 (local address 127.16.235.67:33515)
0806 22:40:04.192247 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.192249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.192261 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:40:04.192319 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.192320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.192332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.192409 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:04.192635 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.193475 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.194991 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.195493 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.195598 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.195609 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.195798 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.195842 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:40:04.195940 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.192078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:33515 (local address 127.16.235.66:56739)
0806 22:40:04.192139 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.192154 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:04.192209 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:04.192470 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:04.192471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:04.192473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:04.192554 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.192556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.193574 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.193575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.194857 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.194861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.195614 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.195616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.195716 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.195721 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:04.195779 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:40:04.195829 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:40:04.196069 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.192349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:42809 (local address 127.16.235.65:43305)
0806 22:40:04.192413 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.192415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.192498 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:40:04.192554 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.192555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.192577 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.192662 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:04.192957 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.193668 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.195008 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.195648 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.195728 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.195823 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.195941 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.195992 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:40:04.196099 5973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.192243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:43305 (local address 127.16.235.66:42809)
0806 22:40:04.192417 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.192428 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:04.192489 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:04.192688 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:04.192688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:04.192690 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:04.192868 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.192869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.193739 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.193740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.194883 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:04.194885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.195742 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.195743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:04.195842 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.195847 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:40:04.195917 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:40:04.195959 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":128,"threads_started":1}
I20260806 22:40:04.196415 5485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c25090e86d04fdfb6066ccd5141df54" candidate_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cc88fd869954bc588eb232dcab876e1" is_pre_election: true
I20260806 22:40:04.196421 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c25090e86d04fdfb6066ccd5141df54" candidate_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" is_pre_election: true
I20260806 22:40:04.196602 5485 raft_consensus.cc:2454] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47acc9cf0f004dae9bbaf7efa3c3440c in term 0.
I20260806 22:40:04.196602 5875 raft_consensus.cc:2454] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47acc9cf0f004dae9bbaf7efa3c3440c in term 0.
I20260806 22:40:04.196861 5588 leader_election.cc:302] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47acc9cf0f004dae9bbaf7efa3c3440c, c584b6ec73e54f8c8fb4d969ae57ee6e; no voters:
I20260806 22:40:04.197005 5970 raft_consensus.cc:2781] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:04.197065 5970 raft_consensus.cc:482] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:04.197094 5970 raft_consensus.cc:3037] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:04.197888 5970 raft_consensus.cc:504] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.198042 5970 leader_election.cc:288] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [CANDIDATE]: Term 1 election: Requested vote from peers c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515), 8cc88fd869954bc588eb232dcab876e1 (127.16.235.65:43305)
I20260806 22:40:04.198207 5875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c25090e86d04fdfb6066ccd5141df54" candidate_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e"
I20260806 22:40:04.198243 5485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c25090e86d04fdfb6066ccd5141df54" candidate_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cc88fd869954bc588eb232dcab876e1"
I20260806 22:40:04.198298 5875 raft_consensus.cc:3037] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:04.198372 5485 raft_consensus.cc:3037] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:04.199175 5485 raft_consensus.cc:2454] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47acc9cf0f004dae9bbaf7efa3c3440c in term 1.
I20260806 22:40:04.199189 5875 raft_consensus.cc:2454] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47acc9cf0f004dae9bbaf7efa3c3440c in term 1.
I20260806 22:40:04.199384 5588 leader_election.cc:302] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47acc9cf0f004dae9bbaf7efa3c3440c, c584b6ec73e54f8c8fb4d969ae57ee6e; no voters:
I20260806 22:40:04.199501 5970 raft_consensus.cc:2781] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:04.199672 5970 raft_consensus.cc:686] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 1 LEADER]: Becoming Leader. State: Replica: 47acc9cf0f004dae9bbaf7efa3c3440c, State: Running, Role: LEADER
I20260806 22:40:04.199787 5970 consensus_queue.cc:237] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.200703 5264 catalog_manager.cc:5495] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c reported cstate change: term changed from 0 to 1, leader changed from <none> to 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66). New cstate: current_term: 1 leader_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:04.221596 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.219378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47982 (local address 127.16.235.66:43145)
0806 22:40:04.219436 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:04.219438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:04.219467 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:40:04.219508 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:04.219509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:04.219529 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:04.219587 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:04.219700 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.220191 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.220917 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:04.221110 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:04.221168 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:04.221294 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:04.221400 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:04.221461 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:04.221491 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 22:40:04.221509 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:40:04.223707 5875 raft_consensus.cc:1261] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 1 FOLLOWER]: Refusing update from remote peer 47acc9cf0f004dae9bbaf7efa3c3440c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:04.223708 5485 raft_consensus.cc:1261] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Refusing update from remote peer 47acc9cf0f004dae9bbaf7efa3c3440c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:04.224123 5970 consensus_queue.cc:1035] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:04.224306 5975 consensus_queue.cc:1035] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:04.226889 5981 mvcc.cc:205] Tried to move back new op lower bound from 7315685393297313792 to 7315685393203027968. Current Snapshot: MvccSnapshot[applied={T|T < 7315685393297313792 or (T in {7315685393297313792})}]
I20260806 22:40:04.227018 5980 mvcc.cc:205] Tried to move back new op lower bound from 7315685393297313792 to 7315685393203027968. Current Snapshot: MvccSnapshot[applied={T|T < 7315685393297313792}]
I20260806 22:40:04.229470 5263 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:04.229581 5263 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:04.230398 5263 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Sending DeleteTablet for 3 replicas of tablet 6c25090e86d04fdfb6066ccd5141df54
I20260806 22:40:04.230803 5445 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c25090e86d04fdfb6066ccd5141df54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:04 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:49822
I20260806 22:40:04.230809 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 5328
I20260806 22:40:04.230851 5848 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c25090e86d04fdfb6066ccd5141df54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:04 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:45416
I20260806 22:40:04.230921 5562 tablet_replica.cc:330] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: stopping tablet replica
I20260806 22:40:04.230974 5562 raft_consensus.cc:2229] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:04.231021 5562 raft_consensus.cc:2258] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:04.231199 5659 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c25090e86d04fdfb6066ccd5141df54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:04 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47980
I20260806 22:40:04.239445 5993 tablet_replica.cc:330] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: stopping tablet replica
I20260806 22:40:04.239624 5993 raft_consensus.cc:2229] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:04.239799 5993 raft_consensus.cc:2258] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:04.240378 5993 ts_tablet_manager.cc:1905] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:04.242296 5993 ts_tablet_manager.cc:1918] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:04.242362 5993 log.cc:1196] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/6c25090e86d04fdfb6066ccd5141df54
I20260806 22:40:04.242586 5993 ts_tablet_manager.cc:1939] T 6c25090e86d04fdfb6066ccd5141df54 P 47acc9cf0f004dae9bbaf7efa3c3440c: Deleting consensus metadata
I20260806 22:40:04.243170 5242 catalog_manager.cc:4841] TS 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145): tablet 6c25090e86d04fdfb6066ccd5141df54 (table test-table [id=d9f00c5df7fd4f4fa5344eea4926bba7]) successfully deleted
I20260806 22:40:04.243397 5992 tablet_replica.cc:330] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: stopping tablet replica
I20260806 22:40:04.243561 5992 raft_consensus.cc:2229] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:04.243747 5992 raft_consensus.cc:2258] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:04.244203 5992 ts_tablet_manager.cc:1905] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:04.245611 5992 ts_tablet_manager.cc:1918] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:04.245703 5992 log.cc:1196] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/6c25090e86d04fdfb6066ccd5141df54
I20260806 22:40:04.246039 5992 ts_tablet_manager.cc:1939] T 6c25090e86d04fdfb6066ccd5141df54 P c584b6ec73e54f8c8fb4d969ae57ee6e: Deleting consensus metadata
I20260806 22:40:04.246538 5244 catalog_manager.cc:4841] TS c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515): tablet 6c25090e86d04fdfb6066ccd5141df54 (table test-table [id=d9f00c5df7fd4f4fa5344eea4926bba7]) successfully deleted
W20260806 22:40:04.247189 5244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:43305 (error 108)
I20260806 22:40:04.247658 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 5555
I20260806 22:40:04.247712 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.247552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:43305 (local address 127.0.0.1:49826)
0806 22:40:04.247612 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.247672 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:40:04.252724 5244 catalog_manager.cc:4584] TS 8cc88fd869954bc588eb232dcab876e1 (127.16.235.65:43305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c25090e86d04fdfb6066ccd5141df54: Network error: Client connection negotiation failed: client connection to 127.16.235.65:43305: connect: Connection refused (error 111)
I20260806 22:40:04.265645 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 5757
I20260806 22:40:04.278927 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 5214
I20260806 22:40:04.291476 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45641
--webserver_interface=127.16.235.126
--webserver_port=43863
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:45641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:04.379161 5998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:04.379428 5998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:04.379453 5998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:04.380743 5998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:04.380810 5998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:04.380824 5998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:04.380837 5998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:04.380849 5998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:04.382773 5998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:45641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=43863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:04.383112 5998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:04.383397 5998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:04.388711 6006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:04.388940 5998 server_base.cc:1034] running on GCE node
W20260806 22:40:04.388736 6009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:04.388726 6007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:04.389364 5998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:04.389737 5998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:04.390915 5998 hybrid_clock.cc:648] HybridClock initialized: now 1786056004390905 us; error 54 us; skew 500 ppm
Aug 06 22:40:04 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056004, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:04.398079 5998 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:40:04.398643 5998 webserver.cc:460] Webserver started at http://127.16.235.126:43863/ using document root <none> and password file <none>
I20260806 22:40:04.398948 5998 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:04.399000 5998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:04.400503 5998 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:04.401305 6019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:04.401544 5998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:04.401650 5998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "26812ef76a8c4a4483204e8b0b66d23a"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "c2b2edf158940773d6c4812f0fe0610f"
server_key_iv: "c39515ccac22d9d980d8f00ade81a003"
server_key_version: "encryptionkey@0"
I20260806 22:40:04.402060 5998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:04.418377 5998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:04.420804 5998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:04.421037 5998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:04.425746 5998 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:45641
I20260806 22:40:04.425784 6101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:45641 every 8 connection(s)
I20260806 22:40:04.426249 5998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:04.427912 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 5998
I20260806 22:40:04.428018 6103 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:40:04.428241 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:43305
--local_ip_for_outbound_sockets=127.16.235.65
--tserver_master_addrs=127.16.235.126:45641
--webserver_port=40515
--webserver_interface=127.16.235.65
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:04.428867 6103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Bootstrap starting.
I20260806 22:40:04.438398 6103 log.cc:824] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:04.440894 6103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:40:04.441258 6103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: Bootstrap complete.
I20260806 22:40:04.443828 6103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:04.444100 6103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26812ef76a8c4a4483204e8b0b66d23a, State: Initialized, Role: FOLLOWER
I20260806 22:40:04.444298 6103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:04.444383 6103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:04.444440 6103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:04.444495 6103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:40:04.445581 6103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:04.445715 6103 leader_election.cc:302] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26812ef76a8c4a4483204e8b0b66d23a; no voters:
I20260806 22:40:04.445935 6103 leader_election.cc:288] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:40:04.446012 6107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:40:04.446168 6107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [term 3 LEADER]: Becoming Leader. State: Replica: 26812ef76a8c4a4483204e8b0b66d23a, State: Running, Role: LEADER
I20260806 22:40:04.446393 6107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } }
I20260806 22:40:04.446241 6103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:04.448304 6108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "26812ef76a8c4a4483204e8b0b66d23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } } }
I20260806 22:40:04.448529 6108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:04.448699 6129 catalog_manager.cc:1259] Loaded cluster ID: 288ee965a3bc4557b44c18b92e40ee89
I20260806 22:40:04.448773 6111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26812ef76a8c4a4483204e8b0b66d23a. Latest consensus state: current_term: 3 leader_uuid: "26812ef76a8c4a4483204e8b0b66d23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26812ef76a8c4a4483204e8b0b66d23a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45641 } } }
I20260806 22:40:04.449029 6111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:04.448975 6129 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: loading cluster ID for follower catalog manager: success
W20260806 22:40:04.449483 6129 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 26812ef76a8c4a4483204e8b0b66d23a: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260806 22:40:04.449556 6129 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260806 22:40:04.449643 6131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:04.450027 6131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:04.450138 6131 catalog_manager.cc:1259] Loaded cluster ID: 288ee965a3bc4557b44c18b92e40ee89
I20260806 22:40:04.450198 6131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 22:40:04.450399 6131 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8848e8e9ad google::LogMessage::Fail() at ??:0
@ 0x7f8848e9188a google::LogMessage::SendToLog() at ??:0
@ 0x7f8848e8e4de google::LogMessage::Flush() at ??:0
@ 0x7f8848e8fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8850db30f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f8850dbc8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f884ad5c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f884ad51954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f884bbb56db start_thread at ??:0
@ 0x7f884798c61f clone at ??:0
W20260806 22:40:04.534834 6105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:04.535075 6105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:04.535104 6105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:04.536351 6105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:04.536407 6105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:04.536437 6105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:40:04.538007 6105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:43305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=40515
--tserver_master_addrs=127.16.235.126:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:04.538296 6105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:04.538581 6105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:04.544167 6161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:04.544180 6160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:04.544180 6165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:04.544567 6105 server_base.cc:1034] running on GCE node
I20260806 22:40:04.544829 6105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:04.545109 6105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:04.546283 6105 hybrid_clock.cc:648] HybridClock initialized: now 1786056004546269 us; error 40 us; skew 500 ppm
Aug 06 22:40:04 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056004, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:04.549139 6105 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:40:04.549691 6105 webserver.cc:460] Webserver started at http://127.16.235.65:40515/ using document root <none> and password file <none>
I20260806 22:40:04.549966 6105 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:04.550041 6105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:04.551640 6105 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:04.552480 6175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:04.552711 6105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:04.552856 6105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "8cc88fd869954bc588eb232dcab876e1"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "ee4c2b00877051eba0e3eaa343afd427"
server_key_iv: "27da23bdf584539f58148627e1bca462"
server_key_version: "encryptionkey@0"
I20260806 22:40:04.553256 6105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:04.569618 6105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:04.572420 6105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:04.572621 6105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:04.572908 6105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:04.573441 6186 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:40:04.575647 6105 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:40:04.575726 6105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:40:04.575776 6105 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:40:04.576510 6105 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:40:04.576573 6105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:04.576658 6186 tablet_bootstrap.cc:492] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Bootstrap starting.
I20260806 22:40:04.584499 6105 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:43305
I20260806 22:40:04.584581 6337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:43305 every 8 connection(s)
I20260806 22:40:04.585145 6105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:04.589130 6186 log.cc:824] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:04.592549 6186 tablet_bootstrap.cc:492] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:40:04.592916 6186 tablet_bootstrap.cc:492] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Bootstrap complete.
I20260806 22:40:04.594365 6186 ts_tablet_manager.cc:1397] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.001s
I20260806 22:40:04.595319 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 6105
I20260806 22:40:04.595631 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:43145
--local_ip_for_outbound_sockets=127.16.235.66
--tserver_master_addrs=127.16.235.126:45641
--webserver_port=35345
--webserver_interface=127.16.235.66
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:04.596062 6186 raft_consensus.cc:348] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.596453 6186 raft_consensus.cc:729] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cc88fd869954bc588eb232dcab876e1, State: Initialized, Role: FOLLOWER
I20260806 22:40:04.596617 6186 consensus_queue.cc:260] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:04.597775 6186 ts_tablet_manager.cc:1428] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1: Time spent starting tablet: real 0.003s user 0.003s sys 0.003s
W20260806 22:40:04.679543 6341 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:40:04.585890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:44933)
0806 22:40:04.586076 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.586125 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:04.586657 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:04.679307 (+ 92650us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
W20260806 22:40:04.679869 6338 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:45641 (0 consecutive failures): Network error: Failed to ping master at 127.16.235.126:45641: Client connection negotiation failed: client connection to 127.16.235.126:45641: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:40:04.680296 6341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.680150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:41501)
0806 22:40:04.680208 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.680272 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:40:04.680661 6341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.680569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:60225)
0806 22:40:04.680613 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.680641 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:40:04.680811 6338 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:40:04.703924 6345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:04.704161 6345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:04.704187 6345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:04.705403 6345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:04.705453 6345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:04.705482 6345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:40:04.707120 6345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:43145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=35345
--tserver_master_addrs=127.16.235.126:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:04.707406 6345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:04.707659 6345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:04.713186 6352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:04.713173 6354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:04.713258 6345 server_base.cc:1034] running on GCE node
W20260806 22:40:04.713151 6351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:04.713593 6345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:04.713840 6345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:04.715000 6345 hybrid_clock.cc:648] HybridClock initialized: now 1786056004714982 us; error 36 us; skew 500 ppm
Aug 06 22:40:04 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056004, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:04.717693 6345 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:40:04.718263 6345 webserver.cc:460] Webserver started at http://127.16.235.66:35345/ using document root <none> and password file <none>
I20260806 22:40:04.718539 6345 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:04.718617 6345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:04.720121 6345 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:04.720779 6366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:04.720954 6345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:04.721067 6345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "47acc9cf0f004dae9bbaf7efa3c3440c"
format_stamp: "Formatted at 2026-08-06 22:40:02 on dist-test-slave-3kk6"
server_key: "29420733466c14b146e35bda0d4dacbc"
server_key_iv: "b7b62ba8b4ac7807578efe1c71ad96b8"
server_key_version: "encryptionkey@0"
I20260806 22:40:04.721457 6345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:04.739719 6345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:04.741870 6345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:04.742044 6345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:04.742305 6345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:04.742700 6345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:04.742767 6345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:04.742812 6345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:04.742861 6345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:04.749367 6345 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:43145
I20260806 22:40:04.749415 6527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:43145 every 8 connection(s)
I20260806 22:40:04.749794 6345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:04.750952 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.750544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:47553)
0806 22:40:04.750716 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.750804 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
W20260806 22:40:04.751387 6530 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:45641 (0 consecutive failures): Network error: Failed to ping master at 127.16.235.126:45641: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
I20260806 22:40:04.751755 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.751596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:44613)
0806 22:40:04.751680 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.751734 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 22:40:04.752206 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.752120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:37069)
0806 22:40:04.752164 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.752190 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:40:04.752451 6530 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:40:04.755854 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 6345
I20260806 22:40:04.756155 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:33515
--local_ip_for_outbound_sockets=127.16.235.67
--tserver_master_addrs=127.16.235.126:45641
--webserver_port=43165
--webserver_interface=127.16.235.67
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:38129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:04.842450 6533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:04.842744 6533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:04.842799 6533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:04.843988 6533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:04.844066 6533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:04.844115 6533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:40:04.845631 6533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:38129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:33515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=43165
--tserver_master_addrs=127.16.235.126:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:04.845917 6533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:04.846167 6533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:04.851576 6538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:04.851581 6544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:04.851576 6540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:04.851837 6533 server_base.cc:1034] running on GCE node
I20260806 22:40:04.852120 6533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:04.852391 6533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:04.853545 6533 hybrid_clock.cc:648] HybridClock initialized: now 1786056004853535 us; error 30 us; skew 500 ppm
Aug 06 22:40:04 dist-test-slave-3kk6 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056004, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:04.856449 6533 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:40:04.856973 6533 webserver.cc:460] Webserver started at http://127.16.235.67:43165/ using document root <none> and password file <none>
I20260806 22:40:04.857257 6533 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:04.857338 6533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:04.858848 6533 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:40:04.859526 6553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:04.859709 6533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:04.859817 6533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e"
format_stamp: "Formatted at 2026-08-06 22:40:03 on dist-test-slave-3kk6"
server_key: "388bacdcc6a6fe26e86af30b95fb8256"
server_key_iv: "6a24076157588f2fdc65880ba4937d24"
server_key_version: "encryptionkey@0"
I20260806 22:40:04.860230 6533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:04.874270 6533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:04.876915 6533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:04.877104 6533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:04.877388 6533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:04.877768 6533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:04.877825 6533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:04.877868 6533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:04.877897 6533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:04.884086 6533 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:33515
I20260806 22:40:04.884136 6716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:33515 every 8 connection(s)
I20260806 22:40:04.884544 6533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:40:04.885756 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.885346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:51883)
0806 22:40:04.885525 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.885610 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
W20260806 22:40:04.886206 6721 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:45641 (0 consecutive failures): Network error: Failed to ping master at 127.16.235.126:45641: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
I20260806 22:40:04.886511 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.886418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:40479)
0806 22:40:04.886465 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.886494 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:40:04.886833 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:04.886756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:46677)
0806 22:40:04.886796 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:04.886819 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:40:04.886955 6721 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:40:04.892508 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 6533
I20260806 22:40:05.681687 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.681338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:35985)
0806 22:40:05.681546 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:05.681605 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
I20260806 22:40:05.753194 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.752907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:56045)
0806 22:40:05.753074 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:05.753127 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260806 22:40:05.887583 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:05.887342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:48191)
0806 22:40:05.887477 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:05.887520 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260806 22:40:06.291234 6735 raft_consensus.cc:482] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:06.291452 6735 raft_consensus.cc:504] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:06.291895 6735 leader_election.cc:288] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515), 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145)
Aug 06 22:40:06 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056004, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.67@KRBTEST.COM
Aug 06 22:40:06 dist-test-slave-3kk6 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056004, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.66@KRBTEST.COM
I20260806 22:40:06.304886 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.292317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:50245 (local address 127.16.235.67:33515)
0806 22:40:06.292499 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.292512 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.292559 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:40:06.293020 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.293022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.293066 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.293224 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:06.294037 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.295161 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.296222 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.296702 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.296834 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.300411 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:06.300417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:06.300417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:06.300433 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:06.302599 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.303234 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.303238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.303238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.303345 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.303668 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.303669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.303669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.304075 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:06.304181 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.304593 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.304762 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":66,"threads_started":1}
I20260806 22:40:06.305032 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.292174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:33515 (local address 127.16.235.65:50245)
0806 22:40:06.292341 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.292360 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.292443 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.293264 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.293267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.293700 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:06.293895 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.293902 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.295278 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.295279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.296124 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.296126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.296825 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.296826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.296952 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.297838 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:06.297849 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:06.300255 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:06.302749 (+ 2494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:06.302760 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:06.302770 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:06.303109 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:06.303430 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:06.303431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:06.303431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:06.303579 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:06.304159 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:06.304183 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:06.304363 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.304503 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.304761 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":59,"thread_start_us":84,"threads_started":1}
I20260806 22:40:06.305205 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.292342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:33969 (local address 127.16.235.66:43145)
0806 22:40:06.292542 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:06.292554 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:06.292577 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:40:06.293020 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:06.293022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:06.293066 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:06.293243 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:06.294028 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.295162 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.296233 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.296719 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.296834 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.300909 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:06.300913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:06.300914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:06.300927 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:06.303129 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.303517 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.303520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.303521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.303644 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:06.303846 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:06.303847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:06.303848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:06.304131 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:06.304209 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:06.304593 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 22:40:06.305073 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":81,"threads_started":1}
I20260806 22:40:06.304840 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.292164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:43145 (local address 127.16.235.65:33969)
0806 22:40:06.292468 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.292481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:06.292551 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:06.293265 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:06.293267 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:06.293706 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:06.293895 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.293902 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.295304 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.295305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.296120 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:06.296123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:06.296945 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:06.296946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:06.297005 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:06.297897 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:06.297902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:06.300780 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:06.303221 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:06.303222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:06.303222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:06.303423 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:06.303722 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:06.303722 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:06.303722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:06.303777 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:06.304201 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:06.304201 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:06.304363 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 22:40:06.304503 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 22:40:06.304707 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":30,"thread_start_us":71,"threads_started":1}
W20260806 22:40:06.306288 6181 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
W20260806 22:40:06.306386 6179 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
I20260806 22:40:06.306493 6179 leader_election.cc:302] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8cc88fd869954bc588eb232dcab876e1; no voters: 47acc9cf0f004dae9bbaf7efa3c3440c, c584b6ec73e54f8c8fb4d969ae57ee6e
I20260806 22:40:06.306613 6735 raft_consensus.cc:2726] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:40:06.682507 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.682356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:56951)
0806 22:40:06.682419 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.682461 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:40:06.753983 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.753730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:44279)
0806 22:40:06.753867 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.753929 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260806 22:40:06.888346 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:06.888117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:33081)
0806 22:40:06.888254 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:06.888302 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260806 22:40:07.683303 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:07.683055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:51597)
0806 22:40:07.683204 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:07.683244 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260806 22:40:07.754927 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:07.754595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:50275)
0806 22:40:07.754821 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:07.754877 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":127,"threads_started":1}
I20260806 22:40:07.889205 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:07.888925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:52013)
0806 22:40:07.889096 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:07.889153 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
I20260806 22:40:08.383495 6747 raft_consensus.cc:482] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:08.383621 6747 raft_consensus.cc:504] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:08.383837 6747 leader_election.cc:288] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515), 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145)
W20260806 22:40:08.384327 6181 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
W20260806 22:40:08.384433 6179 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
I20260806 22:40:08.384529 6179 leader_election.cc:302] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8cc88fd869954bc588eb232dcab876e1; no voters: 47acc9cf0f004dae9bbaf7efa3c3440c, c584b6ec73e54f8c8fb4d969ae57ee6e
I20260806 22:40:08.384622 6747 raft_consensus.cc:2726] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:40:08.684249 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:08.683925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:36567)
0806 22:40:08.684096 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:08.684156 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260806 22:40:08.755761 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:08.755499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:37139)
0806 22:40:08.755646 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:08.755695 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260806 22:40:08.889968 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:08.889725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:54211)
0806 22:40:08.889866 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:08.889908 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260806 22:40:09.685482 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:09.685170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:48463)
0806 22:40:09.685383 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:09.685435 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":145,"threads_started":1}
I20260806 22:40:09.756601 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:09.756345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:45405)
0806 22:40:09.756494 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:09.756551 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260806 22:40:09.890983 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:09.890589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:36641)
0806 22:40:09.890883 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:09.890934 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":223,"threads_started":1}
I20260806 22:40:10.554855 6755 raft_consensus.cc:482] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:10.554998 6755 raft_consensus.cc:504] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:10.555207 6755 leader_election.cc:288] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515), 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145)
W20260806 22:40:10.555655 6179 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
W20260806 22:40:10.555755 6181 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
I20260806 22:40:10.555817 6181 leader_election.cc:302] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8cc88fd869954bc588eb232dcab876e1; no voters: 47acc9cf0f004dae9bbaf7efa3c3440c, c584b6ec73e54f8c8fb4d969ae57ee6e
I20260806 22:40:10.555912 6755 raft_consensus.cc:2726] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:40:10.686177 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:10.685949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:48337)
0806 22:40:10.686080 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:10.686123 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260806 22:40:10.757483 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:10.757178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:37707)
0806 22:40:10.757346 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:10.757412 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260806 22:40:10.891837 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:10.891529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:48571)
0806 22:40:10.891729 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:10.891783 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":88,"threads_started":1}
I20260806 22:40:11.687040 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:11.686794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:55643)
0806 22:40:11.686945 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:11.686989 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260806 22:40:11.758294 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:11.758034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:33953)
0806 22:40:11.758203 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:11.758242 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260806 22:40:11.892714 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:11.892453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:49109)
0806 22:40:11.892608 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:11.892662 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260806 22:40:12.236177 6765 raft_consensus.cc:482] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:12.236299 6765 raft_consensus.cc:504] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:12.236482 6765 leader_election.cc:288] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515), 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145)
W20260806 22:40:12.236884 6181 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
W20260806 22:40:12.236987 6179 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
I20260806 22:40:12.237068 6179 leader_election.cc:302] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8cc88fd869954bc588eb232dcab876e1; no voters: 47acc9cf0f004dae9bbaf7efa3c3440c, c584b6ec73e54f8c8fb4d969ae57ee6e
I20260806 22:40:12.237157 6765 raft_consensus.cc:2726] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:40:12.687855 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:12.687608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:54897)
0806 22:40:12.687759 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:12.687803 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260806 22:40:12.759176 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:12.758886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:46127)
0806 22:40:12.759073 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:12.759120 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260806 22:40:12.893671 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:12.893366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:43097)
0806 22:40:12.893564 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:12.893618 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":121,"threads_started":1}
I20260806 22:40:13.688704 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:13.688450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:49887)
0806 22:40:13.688613 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:13.688655 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260806 22:40:13.760008 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:13.759669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:38255)
0806 22:40:13.759901 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:13.759945 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":141,"threads_started":1}
I20260806 22:40:13.894531 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:13.894281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:60855)
0806 22:40:13.894430 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:13.894481 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260806 22:40:14.689666 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:14.689373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:35713)
0806 22:40:14.689548 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:14.689602 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":76,"threads_started":1}
I20260806 22:40:14.760730 6779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:14.760478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:46611)
0806 22:40:14.760639 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:14.760682 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260806 22:40:14.895418 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:14.895164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:46055)
0806 22:40:14.895331 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:14.895370 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
I20260806 22:40:15.690542 6787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.690244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:56581)
0806 22:40:15.690433 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:15.690491 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":82,"threads_started":1}
I20260806 22:40:15.761546 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.761302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:38025)
0806 22:40:15.761442 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:15.761494 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260806 22:40:15.896162 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:15.895947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:47223)
0806 22:40:15.896076 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:15.896113 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260806 22:40:16.691385 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:16.691134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:46527)
0806 22:40:16.691291 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:16.691332 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
I20260806 22:40:16.762385 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:16.762125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:56945)
0806 22:40:16.762281 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:16.762333 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260806 22:40:16.896927 6793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:16.896673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:41089)
0806 22:40:16.896834 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:16.896877 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260806 22:40:17.692243 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.691976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:56685)
0806 22:40:17.692138 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:17.692190 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260806 22:40:17.744836 6795 raft_consensus.cc:482] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:17.744990 6795 raft_consensus.cc:504] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c584b6ec73e54f8c8fb4d969ae57ee6e" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 33515 } } peers { permanent_uuid: "47acc9cf0f004dae9bbaf7efa3c3440c" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43145 } } peers { permanent_uuid: "8cc88fd869954bc588eb232dcab876e1" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 43305 } }
I20260806 22:40:17.745204 6795 leader_election.cc:288] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515), 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145)
W20260806 22:40:17.745724 6181 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c584b6ec73e54f8c8fb4d969ae57ee6e (127.16.235.67:33515): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
W20260806 22:40:17.745810 6179 leader_election.cc:341] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 47acc9cf0f004dae9bbaf7efa3c3440c (127.16.235.66:43145): Not found: Tablet not found: 6c25090e86d04fdfb6066ccd5141df54
I20260806 22:40:17.745857 6179 leader_election.cc:302] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8cc88fd869954bc588eb232dcab876e1; no voters: 47acc9cf0f004dae9bbaf7efa3c3440c, c584b6ec73e54f8c8fb4d969ae57ee6e
I20260806 22:40:17.745957 6795 raft_consensus.cc:2726] T 6c25090e86d04fdfb6066ccd5141df54 P 8cc88fd869954bc588eb232dcab876e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:40:17.763285 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.763002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:42293)
0806 22:40:17.763140 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:17.763233 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260806 22:40:17.897861 6797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:17.897500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:35047)
0806 22:40:17.897693 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:17.897759 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260806 22:40:18.693068 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:18.692834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:42009)
0806 22:40:18.692981 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:18.693018 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260806 22:40:18.764094 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:18.763848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:44677)
0806 22:40:18.763992 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:18.764043 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260806 22:40:18.898757 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:18.898424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:34995)
0806 22:40:18.898600 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:18.898700 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260806 22:40:19.694168 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.693895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.65:56067)
0806 22:40:19.694064 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.694118 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260806 22:40:19.765022 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.764753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.66:39653)
0806 22:40:19.764923 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.764970 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260806 22:40:19.898028 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 6105
I20260806 22:40:19.902817 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:19.902522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45641 (local address 127.16.235.67:44829)
0806 22:40:19.902705 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:19.902750 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":66,"threads_started":1}
I20260806 22:40:19.911387 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 6345
I20260806 22:40:19.926194 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 6533
2026-08-06T22:40:19Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18850 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:40:20 dist-test-slave-3kk6 krb5kdc[6810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:40:20 dist-test-slave-3kk6 krb5kdc[6810](info): set up 2 sockets
Aug 06 22:40:20 dist-test-slave-3kk6 krb5kdc[6810](info): commencing operation
krb5kdc: starting...
W20260806 22:40:21.189929 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:40:21 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:40:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:40:21.325382 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36595
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:36595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:21.413362 6829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:21.413615 6829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:21.413640 6829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:21.414902 6829 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:40:21.414951 6829 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:40:21.414964 6829 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:40:21.414976 6829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:21.414989 6829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:21.415004 6829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:21.415064 6829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:21.415083 6829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:21.416723 6829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:36595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:21.417021 6829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:21.417320 6829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:21.422943 6835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.422971 6837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.422986 6834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.423010 6829 server_base.cc:1034] running on GCE node
I20260806 22:40:21.423444 6829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:21.423764 6829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:21.424922 6829 hybrid_clock.cc:648] HybridClock initialized: now 1786056021424908 us; error 33 us; skew 500 ppm
Aug 06 22:40:21 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:21.427726 6829 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:40:21.428177 6829 webserver.cc:460] Webserver started at http://127.16.235.126:39287/ using document root <none> and password file <none>
I20260806 22:40:21.428426 6829 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:21.428494 6829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:21.428647 6829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:21.429783 6829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "ac6f179d64c946e787909eab8ceb2a4a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "24795d75e0220704df21ce5456d0f6f2"
server_key_iv: "0d80d97798136f2709508e35eeb1649b"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.430162 6829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "ac6f179d64c946e787909eab8ceb2a4a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "24795d75e0220704df21ce5456d0f6f2"
server_key_iv: "0d80d97798136f2709508e35eeb1649b"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.431723 6829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:21.432495 6850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.432668 6829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:40:21.432778 6829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "ac6f179d64c946e787909eab8ceb2a4a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "24795d75e0220704df21ce5456d0f6f2"
server_key_iv: "0d80d97798136f2709508e35eeb1649b"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.432855 6829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:21.463279 6829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:21.465950 6829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:21.466138 6829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:21.470466 6829 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:36595
I20260806 22:40:21.470510 6927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:36595 every 8 connection(s)
I20260806 22:40:21.470948 6829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:21.472009 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 6829
I20260806 22:40:21.472015 6930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:21.472126 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:40:21.472326 17325 external_mini_cluster.cc:1428] Setting key 0e53775fca082d2ef50be47e7cfadcd8
I20260806 22:40:21.475570 6930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Bootstrap starting.
I20260806 22:40:21.476256 6930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:21.476588 6930 log.cc:824] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:21.477340 6930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: No bootstrap required, opened a new log
Aug 06 22:40:21 dist-test-slave-3kk6 krb5kdc[6810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:21.479691 6930 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:40:21.479899 6930 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:21.479965 6930 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6f179d64c946e787909eab8ceb2a4a, State: Initialized, Role: FOLLOWER
I20260806 22:40:21.480113 6930 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:40:21.480224 6930 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:21.480270 6930 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:21.480324 6930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:21.480902 6930 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:40:21.481032 6930 leader_election.cc:302] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6f179d64c946e787909eab8ceb2a4a; no voters:
I20260806 22:40:21.481248 6930 leader_election.cc:288] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:40:21.481285 6937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:21.481398 6937 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 1 LEADER]: Becoming Leader. State: Replica: ac6f179d64c946e787909eab8ceb2a4a, State: Running, Role: LEADER
I20260806 22:40:21.481554 6930 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:21.481580 6937 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:40:21.482009 6937 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6f179d64c946e787909eab8ceb2a4a. Latest consensus state: current_term: 1 leader_uuid: "ac6f179d64c946e787909eab8ceb2a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } } }
I20260806 22:40:21.482023 6938 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac6f179d64c946e787909eab8ceb2a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } } }
I20260806 22:40:21.482216 6937 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:21.482228 6938 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:21.482808 6949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:21.483500 6949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:21.484436 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.472937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.16.235.126:36595)
0806 22:40:21.473112 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:21.473120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:21.473148 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:40:21.473680 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:21.473689 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:21.473741 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:21.473861 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:21.474750 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.475650 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.476481 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.476943 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.477071 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.479908 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:21.479912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:21.479913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:21.479928 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:21.481985 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:21.482460 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:21.482463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:21.482463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:21.482570 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:21.482888 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:21.482889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:21.482890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:21.483257 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:21.484008 (+ 751us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:21.484080 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:40:21.484235 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260806 22:40:21.485596 6949 catalog_manager.cc:1347] Generated new cluster ID: e42c8b2dc8784192aa6ae51aea75ade1
I20260806 22:40:21.485653 6949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:21.515527 6949 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:40:21.516423 6949 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:21.523020 6949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Generated new TSK 0
I20260806 22:40:21.523259 6949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:40:21.565259 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36595
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:21.652822 6966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:21.653050 6966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:21.653076 6966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:21.654234 6966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:21.654285 6966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:21.654340 6966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:40:21.655923 6966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:21.656196 6966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:21.656456 6966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:21.662009 6978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.662014 6973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.662026 6966 server_base.cc:1034] running on GCE node
W20260806 22:40:21.662009 6971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.662474 6966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:21.662801 6966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:21.663986 6966 hybrid_clock.cc:648] HybridClock initialized: now 1786056021663977 us; error 50 us; skew 500 ppm
Aug 06 22:40:21 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:21.666796 6966 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:40:21.667313 6966 webserver.cc:460] Webserver started at http://127.16.235.65:43889/ using document root <none> and password file <none>
I20260806 22:40:21.667591 6966 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:21.667666 6966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:21.667843 6966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:21.668973 6966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "25a2571380b84f469c3c6c9328a7d30a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "b119b94ff7268f99c59d3321022c7e35"
server_key_iv: "a3c2a6461020a1bd7f0eea393b9ae1b8"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.669369 6966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "25a2571380b84f469c3c6c9328a7d30a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "b119b94ff7268f99c59d3321022c7e35"
server_key_iv: "a3c2a6461020a1bd7f0eea393b9ae1b8"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.670871 6966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:40:21.671689 6988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.671871 6966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:21.671984 6966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "25a2571380b84f469c3c6c9328a7d30a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "b119b94ff7268f99c59d3321022c7e35"
server_key_iv: "a3c2a6461020a1bd7f0eea393b9ae1b8"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.672062 6966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:21.689059 6966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:21.691833 6966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:21.692028 6966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:21.692296 6966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:21.692652 6966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:21.692708 6966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.692745 6966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:21.692795 6966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.699404 6966 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:40533
I20260806 22:40:21.699460 7154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:40533 every 8 connection(s)
I20260806 22:40:21.699841 6966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:21.702042 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 6966
I20260806 22:40:21.702193 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:40:21.702414 17325 external_mini_cluster.cc:1428] Setting key 9b339365dd0ca5b3efb7190b2806541f
Aug 06 22:40:21 dist-test-slave-3kk6 krb5kdc[6810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:21.711118 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.700690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:47921 (local address 127.16.235.126:36595)
0806 22:40:21.700756 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:21.700759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:21.701005 (+ 246us) server_negotiation.cc:407] Connection header received
0806 22:40:21.701526 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:21.701528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:21.701540 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:21.701616 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:21.702780 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.703414 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.704458 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.704802 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.704883 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.707656 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:21.707658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:21.707659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:21.707669 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:21.709490 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:21.709941 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:21.709943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:21.709943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:21.710016 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:21.710279 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:21.710280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:21.710280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:21.710445 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:21.710524 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:21.710855 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 22:40:21.710968 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:40:21.711171 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.700656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:47921)
0806 22:40:21.700832 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:21.700879 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:21.701433 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:21.701674 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:21.701678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:21.702382 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:21.702640 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.702646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.703517 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.703518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.704354 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.704356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.704925 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.704927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.705102 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.705730 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:21.705744 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:21.707544 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:21.709584 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:21.709592 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:21.709594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:21.709856 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:21.710096 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:21.710096 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:21.710097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:21.710204 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:21.710517 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:21.710522 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:21.710704 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 22:40:21.710807 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 22:40:21.710946 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260806 22:40:21.711833 7156 heartbeater.cc:347] Connected to a master server at 127.16.235.126:36595
I20260806 22:40:21.711958 7156 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:21.712188 7156 heartbeater.cc:511] Master 127.16.235.126:36595 requested a full tablet report, sending...
I20260806 22:40:21.712872 6877 ts_manager.cc:194] Registered new tserver with Master: 25a2571380b84f469c3c6c9328a7d30a (127.16.235.65:40533)
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
I20260806 22:40:21.713792 6877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:47921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:40:21.733248 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36595
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:21.819875 7167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:21.820103 7167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:21.820129 7167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:21.821362 7167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:21.821416 7167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:21.821473 7167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:40:21.823125 7167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:21.823395 7167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:21.823653 7167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:21.829109 7174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.829109 7172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.829161 7179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.829401 7167 server_base.cc:1034] running on GCE node
I20260806 22:40:21.829695 7167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:21.829981 7167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:21.831151 7167 hybrid_clock.cc:648] HybridClock initialized: now 1786056021831143 us; error 38 us; skew 500 ppm
Aug 06 22:40:21 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:21.833856 7167 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:40:21.834352 7167 webserver.cc:460] Webserver started at http://127.16.235.66:43879/ using document root <none> and password file <none>
I20260806 22:40:21.834612 7167 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:21.834720 7167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:21.834892 7167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:21.836088 7167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "208237e2644140fea5a3c19ae33db423"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "6a362bc72c3e88e8030e1c98bac1ed17"
server_key_iv: "d8eb34e7285475ccb40d1ae151301c82"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.836552 7167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "208237e2644140fea5a3c19ae33db423"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "6a362bc72c3e88e8030e1c98bac1ed17"
server_key_iv: "d8eb34e7285475ccb40d1ae151301c82"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.838001 7167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:21.838816 7190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.838999 7167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:21.839109 7167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "208237e2644140fea5a3c19ae33db423"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "6a362bc72c3e88e8030e1c98bac1ed17"
server_key_iv: "d8eb34e7285475ccb40d1ae151301c82"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.839201 7167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:21.848531 7167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:21.850558 7167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:21.850791 7167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:21.851083 7167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:21.851438 7167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:21.851497 7167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.851545 7167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:21.851570 7167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.857977 7374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:40747 every 8 connection(s)
I20260806 22:40:21.857986 7167 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:40747
I20260806 22:40:21.858530 7167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:21.859516 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 7167
I20260806 22:40:21.859652 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:40:21.859870 17325 external_mini_cluster.cc:1428] Setting key 401c01ed0614a2c2292436b290ebc73d
Aug 06 22:40:21 dist-test-slave-3kk6 krb5kdc[6810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
I20260806 22:40:21.869550 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.859505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:46659 (local address 127.16.235.126:36595)
0806 22:40:21.859583 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:21.859585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:21.859824 (+ 239us) server_negotiation.cc:407] Connection header received
0806 22:40:21.860526 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:21.860527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:21.860537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:21.860587 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:21.861655 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.862439 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.863604 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.863821 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.863918 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.866498 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:21.866500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:21.866500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:21.866509 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:21.868067 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:21.868494 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:21.868495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:21.868496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:21.868563 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:21.868834 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:21.868835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:21.868835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:21.868967 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:21.869048 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:21.869346 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:40:21.869451 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:40:21.869755 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:21.859471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:46659)
0806 22:40:21.859658 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:21.859711 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:21.860423 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:21.860682 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:21.860687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:21.861339 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:21.861548 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.861553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.862573 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.862574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.863487 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:21.863489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:21.863941 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:21.863942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:21.864088 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:21.864691 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:21.864700 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:21.866385 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:21.868158 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:21.868167 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:21.868169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:21.868412 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:21.868642 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:21.868643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:21.868644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:21.868762 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:21.869039 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:21.869047 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:21.869203 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 22:40:21.869321 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:40:21.869496 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
I20260806 22:40:21.870293 7375 heartbeater.cc:347] Connected to a master server at 127.16.235.126:36595
I20260806 22:40:21.870412 7375 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:21.870796 7375 heartbeater.cc:511] Master 127.16.235.126:36595 requested a full tablet report, sending...
I20260806 22:40:21.871253 6877 ts_manager.cc:194] Registered new tserver with Master: 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747)
I20260806 22:40:21.871630 6877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:46659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:40:21.889685 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36595
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:21.977123 7382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:21.977370 7382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:21.977396 7382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:21.978642 7382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:21.978786 7382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:21.978840 7382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:40:21.980458 7382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:21.980741 7382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:21.981017 7382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:21.986545 7390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:21.986568 7395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.986690 7382 server_base.cc:1034] running on GCE node
W20260806 22:40:21.986557 7392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:21.987079 7382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:21.987365 7382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:21.988526 7382 hybrid_clock.cc:648] HybridClock initialized: now 1786056021988514 us; error 33 us; skew 500 ppm
Aug 06 22:40:21 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:21.991263 7382 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:40:21.991788 7382 webserver.cc:460] Webserver started at http://127.16.235.67:39181/ using document root <none> and password file <none>
I20260806 22:40:21.992013 7382 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:21.992056 7382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:21.992156 7382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:40:21.993223 7382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "c1187f9ffd8d4f3aa007da32a81b78ed"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "a451037ac2ddb0d77e9d34caffa36107"
server_key_iv: "b9e03bf6f5241d93176ebe55c7097c9e"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.993556 7382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "c1187f9ffd8d4f3aa007da32a81b78ed"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "a451037ac2ddb0d77e9d34caffa36107"
server_key_iv: "b9e03bf6f5241d93176ebe55c7097c9e"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.994968 7382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:40:21.995739 7404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:21.995952 7382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:21.996052 7382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "c1187f9ffd8d4f3aa007da32a81b78ed"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "a451037ac2ddb0d77e9d34caffa36107"
server_key_iv: "b9e03bf6f5241d93176ebe55c7097c9e"
server_key_version: "encryptionkey@0"
I20260806 22:40:21.996114 7382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:22.006810 7382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:22.009039 7382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:22.009186 7382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:22.009409 7382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:22.009770 7382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:22.009807 7382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:22.009830 7382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:22.009843 7382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:22.016016 7382 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:43969
I20260806 22:40:22.016096 7578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:43969 every 8 connection(s)
I20260806 22:40:22.016472 7382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:40:22 dist-test-slave-3kk6 krb5kdc[6810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056021, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:22.026254 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 7382
I20260806 22:40:22.026373 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:40:22.026592 17325 external_mini_cluster.cc:1428] Setting key 8e7b2950e8f79afd54b71ee0d5894b2d
I20260806 22:40:22.026940 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.017352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:36087 (local address 127.16.235.126:36595)
0806 22:40:22.017426 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.017429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.017682 (+ 253us) server_negotiation.cc:407] Connection header received
0806 22:40:22.018219 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.018220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.018232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.018300 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:22.019440 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.020011 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.021006 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.021201 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.021255 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.023764 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:22.023766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:22.023766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:22.023774 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:22.025158 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:22.025736 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:22.025737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:22.025737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:22.025791 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:22.026084 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:22.026085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:22.026086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:22.026266 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:22.026360 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.026727 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.026828 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:40:22.027160 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.017338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:36087)
0806 22:40:22.017552 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:22.017591 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:22.018089 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:22.018411 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:22.018415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:22.019113 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:22.019328 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.019333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.020099 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.020100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.020925 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.020927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.021277 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.021277 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.021381 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.021990 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:22.022000 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:22.023658 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:22.025394 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:22.025400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:22.025402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:22.025634 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:22.025859 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:22.025860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:22.025860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:22.025970 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:22.026370 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:22.026377 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:22.026534 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 22:40:22.026631 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:40:22.026890 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":122,"threads_started":1}
I20260806 22:40:22.027726 7579 heartbeater.cc:347] Connected to a master server at 127.16.235.126:36595
I20260806 22:40:22.027851 7579 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:22.028084 7579 heartbeater.cc:511] Master 127.16.235.126:36595 requested a full tablet report, sending...
I20260806 22:40:22.028447 6877 ts_manager.cc:194] Registered new tserver with Master: c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
I20260806 22:40:22.028855 6877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:36087
I20260806 22:40:22.039057 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:40:22.048099 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.040678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.16.235.126:36595)
0806 22:40:22.040750 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.040753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.040879 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:40:22.040972 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.040973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.040993 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.041064 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:22.041808 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.042471 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.043476 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.043701 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.043790 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.045173 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:22.045175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:22.045176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:22.045186 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:22.046627 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:22.047097 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:22.047098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:22.047099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:22.047174 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:22.047415 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:22.047416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:22.047416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:22.047598 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:22.047738 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.047877 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.047975 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:40:22.049887 6877 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:22.051770 6877 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:22.059051 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.054494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42286 (local address 127.16.235.66:40747)
0806 22:40:22.054674 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.054680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.054876 (+ 196us) server_negotiation.cc:407] Connection header received
0806 22:40:22.055039 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.055044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.055064 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.055166 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:22.055472 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.056629 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.057899 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.058549 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.058641 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.058726 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.058886 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.058951 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260806 22:40:22.059072 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.054687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54180 (local address 127.16.235.67:43969)
0806 22:40:22.054933 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.054937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.055116 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:40:22.055164 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.055166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.055182 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.055269 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:22.055491 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.056624 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.057963 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.058727 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.058809 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.058862 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.058950 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.058991 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":98,"threads_started":1}
I20260806 22:40:22.059051 7591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.054395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:40533 (local address 127.0.0.1:59562)
0806 22:40:22.054876 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:22.054887 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:22.054943 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:22.055365 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:22.055366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:22.055370 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:22.055435 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.055437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.056348 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.056349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.057631 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.057634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.058754 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.058755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.058824 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.058828 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:22.058879 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:40:22.058953 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":112,"threads_started":1}
I20260806 22:40:22.059062 7593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.054395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:43969 (local address 127.0.0.1:54180)
0806 22:40:22.055034 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:22.055044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:22.055096 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:22.055361 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:22.055361 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:22.055363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:22.055411 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.055413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.056743 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.056744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.057884 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.057887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.058810 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.058811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.058877 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.058881 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:22.058936 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:40:22.058977 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":36,"thread_start_us":98,"threads_started":1}
I20260806 22:40:22.059574 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.054383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:40747 (local address 127.0.0.1:42286)
0806 22:40:22.054742 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:22.054805 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:22.054896 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:22.055270 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:22.055273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:22.055275 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:22.055356 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.055357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.056763 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.056764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.057800 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.057803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.058643 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.058644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.058750 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.058765 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:40:22.059447 (+ 682us) client_negotiation.cc:240] Negotiation successful
0806 22:40:22.059502 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":31,"thread_start_us":74,"threads_started":1}
I20260806 22:40:22.059739 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.054693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59562 (local address 127.16.235.65:40533)
0806 22:40:22.055035 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.055039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.055051 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:40:22.055098 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.055100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.055188 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.055267 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:22.055502 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.056232 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.057732 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.058550 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.058625 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.058712 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.059594 (+ 882us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.059644 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":101,"threads_started":1}
I20260806 22:40:22.060035 7475 tablet_service.cc:1467] Processing CreateTablet for tablet d9da11a6f61148489cc690c79ec4724a (DEFAULT_TABLE table=test-table [id=af87ca8db5004374b7e1cf5ccfdd24b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:22.060287 7271 tablet_service.cc:1467] Processing CreateTablet for tablet d9da11a6f61148489cc690c79ec4724a (DEFAULT_TABLE table=test-table [id=af87ca8db5004374b7e1cf5ccfdd24b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:22.060451 7475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9da11a6f61148489cc690c79ec4724a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:22.060549 7271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9da11a6f61148489cc690c79ec4724a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:22.060498 7055 tablet_service.cc:1467] Processing CreateTablet for tablet d9da11a6f61148489cc690c79ec4724a (DEFAULT_TABLE table=test-table [id=af87ca8db5004374b7e1cf5ccfdd24b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:22.060804 7055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9da11a6f61148489cc690c79ec4724a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:22.063164 7599 tablet_bootstrap.cc:492] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Bootstrap starting.
I20260806 22:40:22.063832 7599 tablet_bootstrap.cc:654] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:22.063948 7600 tablet_bootstrap.cc:492] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: Bootstrap starting.
I20260806 22:40:22.064496 7599 log.cc:824] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:22.064594 7602 tablet_bootstrap.cc:492] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Bootstrap starting.
I20260806 22:40:22.064635 7600 tablet_bootstrap.cc:654] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:22.065143 7600 log.cc:824] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:22.065277 7602 tablet_bootstrap.cc:654] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:22.065440 7599 tablet_bootstrap.cc:492] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: No bootstrap required, opened a new log
I20260806 22:40:22.065546 7599 ts_tablet_manager.cc:1397] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:22.065833 7602 log.cc:824] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:22.065974 7600 tablet_bootstrap.cc:492] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: No bootstrap required, opened a new log
I20260806 22:40:22.066048 7600 ts_tablet_manager.cc:1397] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:22.067258 7602 tablet_bootstrap.cc:492] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: No bootstrap required, opened a new log
I20260806 22:40:22.067333 7602 ts_tablet_manager.cc:1397] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:40:22.067718 7599 raft_consensus.cc:348] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:22.067974 7599 raft_consensus.cc:374] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:22.068018 7599 raft_consensus.cc:729] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 208237e2644140fea5a3c19ae33db423, State: Initialized, Role: FOLLOWER
I20260806 22:40:22.068149 7599 consensus_queue.cc:260] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:22.068075 7600 raft_consensus.cc:348] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:22.068229 7600 raft_consensus.cc:374] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:22.068266 7600 raft_consensus.cc:729] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1187f9ffd8d4f3aa007da32a81b78ed, State: Initialized, Role: FOLLOWER
I20260806 22:40:22.068406 7600 consensus_queue.cc:260] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:22.068482 7599 ts_tablet_manager.cc:1428] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:40:22.068490 7375 heartbeater.cc:503] Master 127.16.235.126:36595 was elected leader, sending a full tablet report...
I20260806 22:40:22.068706 7600 ts_tablet_manager.cc:1428] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:40:22.068709 7579 heartbeater.cc:503] Master 127.16.235.126:36595 was elected leader, sending a full tablet report...
I20260806 22:40:22.069195 7602 raft_consensus.cc:348] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:22.069365 7602 raft_consensus.cc:374] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:22.069411 7602 raft_consensus.cc:729] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a2571380b84f469c3c6c9328a7d30a, State: Initialized, Role: FOLLOWER
I20260806 22:40:22.069526 7602 consensus_queue.cc:260] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:22.069867 7602 ts_tablet_manager.cc:1428] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:22.070051 7156 heartbeater.cc:503] Master 127.16.235.126:36595 was elected leader, sending a full tablet report...
W20260806 22:40:22.109386 7376 tablet.cc:2369] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:22.200657 7162 tablet.cc:2369] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:22.267146 7581 tablet.cc:2369] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:22.454463 7608 raft_consensus.cc:482] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:22.454723 7608 raft_consensus.cc:504] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:22.455219 7608 leader_election.cc:288] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25a2571380b84f469c3c6c9328a7d30a (127.16.235.65:40533), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
I20260806 22:40:22.459690 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.455566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:38863 (local address 127.16.235.65:40533)
0806 22:40:22.455627 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.455630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.455760 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:40:22.455807 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.455808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.455823 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.455898 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:22.456143 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.456985 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.458637 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.459264 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.459353 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.459363 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.459566 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.459611 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:40:22.459697 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.455608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:36725 (local address 127.16.235.67:43969)
0806 22:40:22.455674 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.455676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.455758 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:40:22.455808 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.455808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.455820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.455898 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:22.456143 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.457007 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.458636 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.459264 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.459354 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.459363 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.459566 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.459611 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:40:22.459748 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.455466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:43969 (local address 127.16.235.66:36725)
0806 22:40:22.455676 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:22.455688 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:22.455749 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:22.455919 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:22.455920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:22.455922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:22.456057 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.456059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.457094 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.457095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.458483 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.458487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.459372 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.459373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.459467 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.459476 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:40:22.459542 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:40:22.459615 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":96,"thread_start_us":102,"threads_started":1}
I20260806 22:40:22.459750 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.455465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:40533 (local address 127.16.235.66:38863)
0806 22:40:22.455651 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:22.455665 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:22.455749 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:22.455923 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:22.455924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:22.455925 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:22.456058 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.456059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.457095 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.457095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.458483 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:22.458487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.459372 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.459373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:22.459467 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.459474 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:40:22.459542 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:40:22.459615 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":78,"thread_start_us":108,"threads_started":1}
I20260806 22:40:22.460245 7090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da11a6f61148489cc690c79ec4724a" candidate_uuid: "208237e2644140fea5a3c19ae33db423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a2571380b84f469c3c6c9328a7d30a" is_pre_election: true
I20260806 22:40:22.460208 7504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da11a6f61148489cc690c79ec4724a" candidate_uuid: "208237e2644140fea5a3c19ae33db423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" is_pre_election: true
I20260806 22:40:22.460412 7090 raft_consensus.cc:2454] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 208237e2644140fea5a3c19ae33db423 in term 0.
I20260806 22:40:22.460412 7504 raft_consensus.cc:2454] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 208237e2644140fea5a3c19ae33db423 in term 0.
I20260806 22:40:22.460654 7198 leader_election.cc:302] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 208237e2644140fea5a3c19ae33db423, c1187f9ffd8d4f3aa007da32a81b78ed; no voters:
I20260806 22:40:22.460888 7608 raft_consensus.cc:2781] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:22.460944 7608 raft_consensus.cc:482] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:22.460961 7608 raft_consensus.cc:3037] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:22.461796 7608 raft_consensus.cc:504] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:22.461963 7608 leader_election.cc:288] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [CANDIDATE]: Term 1 election: Requested vote from peers 25a2571380b84f469c3c6c9328a7d30a (127.16.235.65:40533), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
I20260806 22:40:22.462194 7090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da11a6f61148489cc690c79ec4724a" candidate_uuid: "208237e2644140fea5a3c19ae33db423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a2571380b84f469c3c6c9328a7d30a"
I20260806 22:40:22.462193 7504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9da11a6f61148489cc690c79ec4724a" candidate_uuid: "208237e2644140fea5a3c19ae33db423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed"
I20260806 22:40:22.462282 7504 raft_consensus.cc:3037] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:22.462282 7090 raft_consensus.cc:3037] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:22.463254 7504 raft_consensus.cc:2454] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 208237e2644140fea5a3c19ae33db423 in term 1.
I20260806 22:40:22.463254 7090 raft_consensus.cc:2454] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 208237e2644140fea5a3c19ae33db423 in term 1.
I20260806 22:40:22.463454 7198 leader_election.cc:302] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 208237e2644140fea5a3c19ae33db423, c1187f9ffd8d4f3aa007da32a81b78ed; no voters:
I20260806 22:40:22.463614 7608 raft_consensus.cc:2781] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:22.463807 7608 raft_consensus.cc:686] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 1 LEADER]: Becoming Leader. State: Replica: 208237e2644140fea5a3c19ae33db423, State: Running, Role: LEADER
I20260806 22:40:22.463917 7608 consensus_queue.cc:237] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:22.464874 6877 catalog_manager.cc:5495] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 208237e2644140fea5a3c19ae33db423 (127.16.235.66). New cstate: current_term: 1 leader_uuid: "208237e2644140fea5a3c19ae33db423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } health_report { overall_health: UNKNOWN } } }
I20260806 22:40:22.505419 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.503110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.16.235.66:40747)
0806 22:40:22.503172 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:22.503173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:22.503183 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:40:22.503241 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:22.503242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:22.503264 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:22.503331 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:22.503465 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.503956 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.504671 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:22.504866 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:22.504917 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:22.505077 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:22.505189 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:22.505252 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:22.505285 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:40:22.505304 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:40:22.507596 7504 raft_consensus.cc:1261] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [term 1 FOLLOWER]: Refusing update from remote peer 208237e2644140fea5a3c19ae33db423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:22.507596 7090 raft_consensus.cc:1261] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Refusing update from remote peer 208237e2644140fea5a3c19ae33db423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:22.508011 7608 consensus_queue.cc:1035] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:22.508307 7613 consensus_queue.cc:1035] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:22.511215 7619 mvcc.cc:205] Tried to move back new op lower bound from 7315685468187541504 to 7315685468012965888. Current Snapshot: MvccSnapshot[applied={T|T < 7315685468187541504}]
I20260806 22:40:22.511253 7617 mvcc.cc:205] Tried to move back new op lower bound from 7315685468187541504 to 7315685468012965888. Current Snapshot: MvccSnapshot[applied={T|T < 7315685468187541504}]
I20260806 22:40:22.513644 6875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:22.513758 6875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:22.514605 6875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Sending DeleteTablet for 3 replicas of tablet d9da11a6f61148489cc690c79ec4724a
I20260806 22:40:22.515064 7055 tablet_service.cc:1514] Processing DeleteTablet for tablet d9da11a6f61148489cc690c79ec4724a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:22 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:59562
I20260806 22:40:22.515054 7271 tablet_service.cc:1514] Processing DeleteTablet for tablet d9da11a6f61148489cc690c79ec4724a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:22 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:42286
I20260806 22:40:22.515306 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 6966
I20260806 22:40:22.515389 7627 tablet_replica.cc:330] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: stopping tablet replica
I20260806 22:40:22.515377 7475 tablet_service.cc:1514] Processing DeleteTablet for tablet d9da11a6f61148489cc690c79ec4724a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:22 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:54180
I20260806 22:40:22.515496 7627 raft_consensus.cc:2229] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:22.515614 7627 raft_consensus.cc:2258] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:22.527681 7628 tablet_replica.cc:330] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: stopping tablet replica
I20260806 22:40:22.527878 7628 raft_consensus.cc:2229] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:40:22.528060 7628 raft_consensus.cc:2258] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:22.528095 7629 tablet_replica.cc:330] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: stopping tablet replica
I20260806 22:40:22.528270 7629 raft_consensus.cc:2229] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 22:40:22.528330 6855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:40533 (error 108)
I20260806 22:40:22.528438 7629 raft_consensus.cc:2258] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:22.528649 7628 ts_tablet_manager.cc:1905] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:22.528960 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.528822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:40533 (local address 127.0.0.1:59574)
0806 22:40:22.528880 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:22.528921 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:40:22.529201 7629 ts_tablet_manager.cc:1905] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 22:40:22.529179 6855 catalog_manager.cc:4584] TS 25a2571380b84f469c3c6c9328a7d30a (127.16.235.65:40533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9da11a6f61148489cc690c79ec4724a: Network error: Client connection negotiation failed: client connection to 127.16.235.65:40533: connect: Connection refused (error 111)
I20260806 22:40:22.529748 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 7167
I20260806 22:40:22.531453 7629 ts_tablet_manager.cc:1918] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:22.531539 7629 log.cc:1196] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/d9da11a6f61148489cc690c79ec4724a
I20260806 22:40:22.531908 7629 ts_tablet_manager.cc:1939] T d9da11a6f61148489cc690c79ec4724a P c1187f9ffd8d4f3aa007da32a81b78ed: Deleting consensus metadata
I20260806 22:40:22.532413 6856 catalog_manager.cc:4841] TS c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969): tablet d9da11a6f61148489cc690c79ec4724a (table test-table [id=af87ca8db5004374b7e1cf5ccfdd24b5]) successfully deleted
I20260806 22:40:22.541828 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 7382
I20260806 22:40:22.541956 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:22.541787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:40747 (local address 127.0.0.1:42314)
0806 22:40:22.541860 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:22.541910 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.66:40747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:40:22.550940 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 6829
I20260806 22:40:42.561530 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36595
--webserver_interface=127.16.235.126
--webserver_port=39287
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:36595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:40:42.649391 7633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:42.649636 7633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:42.649660 7633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:42.650852 7633 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:40:42.650900 7633 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:40:42.650914 7633 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:40:42.650926 7633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:40:42.650939 7633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:42.650952 7633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:42.650964 7633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:40:42.650976 7633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:40:42.652513 7633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:36595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=39287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:42.652734 7633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:42.652935 7633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:40:42.663012 7633 server_base.cc:1034] running on GCE node
W20260806 22:40:42.663110 7641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:42.663009 7645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:42.663062 7643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:42.663633 7633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:42.664004 7633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:42.665174 7633 hybrid_clock.cc:648] HybridClock initialized: now 1786056042665162 us; error 47 us; skew 500 ppm
Aug 06 22:40:42 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:42.671825 7633 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:40:42.672430 7633 webserver.cc:460] Webserver started at http://127.16.235.126:39287/ using document root <none> and password file <none>
I20260806 22:40:42.672696 7633 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:42.672809 7633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:42.674350 7633 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:40:42.675314 7659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:42.675532 7633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:40:42.675662 7633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "ac6f179d64c946e787909eab8ceb2a4a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "24795d75e0220704df21ce5456d0f6f2"
server_key_iv: "0d80d97798136f2709508e35eeb1649b"
server_key_version: "encryptionkey@0"
I20260806 22:40:42.676173 7633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:42.700580 7633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:42.703114 7633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:42.703315 7633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:42.707916 7633 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:36595
I20260806 22:40:42.707959 7726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:36595 every 8 connection(s)
I20260806 22:40:42.708439 7633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:40:42.708773 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 7633
I20260806 22:40:42.709133 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:40533
--local_ip_for_outbound_sockets=127.16.235.65
--tserver_master_addrs=127.16.235.126:36595
--webserver_port=43889
--webserver_interface=127.16.235.65
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:42.710310 7727 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:40:42.711134 7727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Bootstrap starting.
I20260806 22:40:42.720582 7727 log.cc:824] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:42.722906 7727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:40:42.723256 7727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Bootstrap complete.
I20260806 22:40:42.725736 7727 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:40:42.726007 7727 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6f179d64c946e787909eab8ceb2a4a, State: Initialized, Role: FOLLOWER
I20260806 22:40:42.726202 7727 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:40:42.726284 7727 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:40:42.726344 7727 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:40:42.726404 7727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:40:42.727592 7727 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:40:42.727723 7727 leader_election.cc:302] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6f179d64c946e787909eab8ceb2a4a; no voters:
I20260806 22:40:42.727948 7727 leader_election.cc:288] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:40:42.727989 7737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:40:42.728114 7737 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 2 LEADER]: Becoming Leader. State: Replica: ac6f179d64c946e787909eab8ceb2a4a, State: Running, Role: LEADER
I20260806 22:40:42.728253 7727 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:40:42.728271 7737 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:40:42.728700 7738 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ac6f179d64c946e787909eab8ceb2a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } } }
I20260806 22:40:42.728719 7739 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6f179d64c946e787909eab8ceb2a4a. Latest consensus state: current_term: 2 leader_uuid: "ac6f179d64c946e787909eab8ceb2a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } } }
I20260806 22:40:42.728789 7738 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:42.728814 7739 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: This master's current role is: LEADER
I20260806 22:40:42.729035 7746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:40:42.730011 7746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:40:42.730192 7746 catalog_manager.cc:1259] Loaded cluster ID: e42c8b2dc8784192aa6ae51aea75ade1
I20260806 22:40:42.730394 7746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:40:42.731313 7746 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:40:42.731519 7746 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Loaded TSK: 0
I20260806 22:40:42.737267 7746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Generated new TSK 1
I20260806 22:40:42.737381 7746 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:40:42.809510 7731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:42.809789 7731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:42.809836 7731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:42.811136 7731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:42.811210 7731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:42.811259 7731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:40:42.812832 7731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:40533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=43889
--tserver_master_addrs=127.16.235.126:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:42.813125 7731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:42.813385 7731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:42.819687 7773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:42.819725 7775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:42.819694 7771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:42.820025 7731 server_base.cc:1034] running on GCE node
I20260806 22:40:42.820241 7731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:42.820559 7731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:42.821756 7731 hybrid_clock.cc:648] HybridClock initialized: now 1786056042821739 us; error 30 us; skew 500 ppm
Aug 06 22:40:42 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:42.824988 7731 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:40:42.825565 7731 webserver.cc:460] Webserver started at http://127.16.235.65:43889/ using document root <none> and password file <none>
I20260806 22:40:42.825825 7731 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:42.825906 7731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:42.827525 7731 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:42.828413 7787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:42.828572 7731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:42.828667 7731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "25a2571380b84f469c3c6c9328a7d30a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "b119b94ff7268f99c59d3321022c7e35"
server_key_iv: "a3c2a6461020a1bd7f0eea393b9ae1b8"
server_key_version: "encryptionkey@0"
I20260806 22:40:42.829059 7731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:42.837440 7731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:42.839965 7731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:42.840139 7731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:42.840466 7731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:42.840957 7796 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:40:42.843031 7731 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:40:42.843122 7731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:40:42.843204 7731 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:40:42.843897 7731 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:40:42.843966 7731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:42.844027 7796 tablet_bootstrap.cc:492] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Bootstrap starting.
I20260806 22:40:42.851868 7731 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:40533
I20260806 22:40:42.851918 7953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:40533 every 8 connection(s)
I20260806 22:40:42.852412 7731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:40:42.855598 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 7731
I20260806 22:40:42.855780 7796 log.cc:824] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:42.855937 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:40747
--local_ip_for_outbound_sockets=127.16.235.66
--tserver_master_addrs=127.16.235.126:36595
--webserver_port=43879
--webserver_interface=127.16.235.66
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:40:42.858583 7796 tablet_bootstrap.cc:492] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:40:42.858956 7796 tablet_bootstrap.cc:492] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Bootstrap complete.
I20260806 22:40:42.860401 7796 ts_tablet_manager.cc:1397] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20260806 22:40:42.861867 7796 raft_consensus.cc:348] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
Aug 06 22:40:42 dist-test-slave-3kk6 krb5kdc[6810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:42.862785 7796 raft_consensus.cc:729] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a2571380b84f469c3c6c9328a7d30a, State: Initialized, Role: FOLLOWER
I20260806 22:40:42.863037 7796 consensus_queue.cc:260] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } }
I20260806 22:40:42.863451 7796 ts_tablet_manager.cc:1428] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260806 22:40:42.867875 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:42.853257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:50119)
0806 22:40:42.853904 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:42.853952 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:42.854494 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:42.855008 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:42.855013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:42.855324 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:42.855647 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.855667 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.857415 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.857416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.858596 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.858599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.859842 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.859843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.859981 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.860734 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:42.860744 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:42.862792 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:42.865575 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:42.865584 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:42.865586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:42.865940 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:42.866297 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:42.866298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:42.866299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:42.866449 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:42.867042 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:42.867050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:42.867248 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 22:40:42.867376 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:40:42.867591 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":111,"threads_started":1}
I20260806 22:40:42.868705 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:42.853689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:50119 (local address 127.16.235.126:36595)
0806 22:40:42.853875 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:42.853886 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:42.854077 (+ 191us) server_negotiation.cc:407] Connection header received
0806 22:40:42.854768 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:42.854774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:42.854819 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:42.854959 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:42.855828 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.857259 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.859022 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.859703 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.860353 (+ 650us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.862951 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:42.862957 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:42.862957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:42.862973 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:42.865454 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:42.866049 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:42.866052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:42.866053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:42.866208 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:42.866534 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:42.866535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:42.866536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:42.866916 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:42.867008 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:42.868277 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0806 22:40:42.868456 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260806 22:40:42.869506 7954 heartbeater.cc:347] Connected to a master server at 127.16.235.126:36595
I20260806 22:40:42.869626 7954 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:42.869827 7954 heartbeater.cc:511] Master 127.16.235.126:36595 requested a full tablet report, sending...
I20260806 22:40:42.870718 7680 ts_manager.cc:194] Registered new tserver with Master: 25a2571380b84f469c3c6c9328a7d30a (127.16.235.65:40533)
I20260806 22:40:42.871973 7680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:50119
I20260806 22:40:42.873046 7954 heartbeater.cc:503] Master 127.16.235.126:36595 was elected leader, sending a full tablet report...
I20260806 22:40:42.877804 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:42.871439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:40533 (local address 127.0.0.1:46762)
0806 22:40:42.872284 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:42.872328 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:42.872414 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:42.873663 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:42.873666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:42.873668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:42.873756 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.873758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.875057 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.875059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.876448 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.876451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.877437 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.877438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.877553 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.877569 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:40:42.877648 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:40:42.877707 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":139,"threads_started":1}
I20260806 22:40:42.877832 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:42.871780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46762 (local address 127.16.235.65:40533)
0806 22:40:42.871966 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:42.871972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:42.873247 (+ 1275us) server_negotiation.cc:407] Connection header received
0806 22:40:42.873288 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:42.873290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:42.873486 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:42.873564 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:42.873853 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.874951 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.876562 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.877340 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.877422 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.877480 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:42.877690 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:40:42.877743 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260806 22:40:42.878466 7859 tablet_service.cc:1514] Processing DeleteTablet for tablet d9da11a6f61148489cc690c79ec4724a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:22 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:46762
I20260806 22:40:42.879397 7966 tablet_replica.cc:330] stopping tablet replica
I20260806 22:40:42.879515 7966 raft_consensus.cc:2229] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:42.879627 7966 raft_consensus.cc:2258] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:42.880059 7966 ts_tablet_manager.cc:1905] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:42.883028 7966 ts_tablet_manager.cc:1918] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:42.883106 7966 log.cc:1196] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal/wals/d9da11a6f61148489cc690c79ec4724a
I20260806 22:40:42.883361 7966 ts_tablet_manager.cc:1939] T d9da11a6f61148489cc690c79ec4724a P 25a2571380b84f469c3c6c9328a7d30a: Deleting consensus metadata
I20260806 22:40:42.883934 7664 catalog_manager.cc:4841] TS 25a2571380b84f469c3c6c9328a7d30a (127.16.235.65:40533): tablet d9da11a6f61148489cc690c79ec4724a (table test-table [id=af87ca8db5004374b7e1cf5ccfdd24b5]) successfully deleted
W20260806 22:40:42.951288 7961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:42.951567 7961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:42.951611 7961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:42.952847 7961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:42.952926 7961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:42.952975 7961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:40:42.954618 7961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:40747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=43879
--tserver_master_addrs=127.16.235.126:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:42.954957 7961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:42.955214 7961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:42.960626 7977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:42.960678 7975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:42.960793 7961 server_base.cc:1034] running on GCE node
W20260806 22:40:42.960695 7974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:42.961117 7961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:42.961396 7961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:42.962556 7961 hybrid_clock.cc:648] HybridClock initialized: now 1786056042962535 us; error 44 us; skew 500 ppm
Aug 06 22:40:42 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:42.965425 7961 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:40:42.965927 7961 webserver.cc:460] Webserver started at http://127.16.235.66:43879/ using document root <none> and password file <none>
I20260806 22:40:42.966122 7961 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:42.966184 7961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:42.967633 7961 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:42.968437 7987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:42.968640 7961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:42.968729 7961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "208237e2644140fea5a3c19ae33db423"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "6a362bc72c3e88e8030e1c98bac1ed17"
server_key_iv: "d8eb34e7285475ccb40d1ae151301c82"
server_key_version: "encryptionkey@0"
I20260806 22:40:42.969102 7961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:42.977559 7961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:42.979732 7961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:42.979887 7961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:42.980134 7961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:42.980579 7998 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260806 22:40:42.981988 7998 ts_tablet_manager.cc:1852] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 22:40:42.982266 7998 ts_tablet_manager.cc:1905] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:42.984357 7998 ts_tablet_manager.cc:1918] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:42.984467 7998 log.cc:1196] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/d9da11a6f61148489cc690c79ec4724a
I20260806 22:40:42.984763 7998 ts_tablet_manager.cc:1939] T d9da11a6f61148489cc690c79ec4724a P 208237e2644140fea5a3c19ae33db423: Deleting consensus metadata
I20260806 22:40:42.984964 7961 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 22:40:42.985020 7961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 22:40:42.985047 7961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:42.985090 7961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:42.991407 7961 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:40747
I20260806 22:40:42.991468 8145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:40747 every 8 connection(s)
I20260806 22:40:42.991808 7961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:40:42.994918 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 7961
I20260806 22:40:42.995245 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:43969
--local_ip_for_outbound_sockets=127.16.235.67
--tserver_master_addrs=127.16.235.126:36595
--webserver_port=39181
--webserver_interface=127.16.235.67
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:40:42 dist-test-slave-3kk6 krb5kdc[6810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056042, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:43.002458 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:42.992542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:60699 (local address 127.16.235.126:36595)
0806 22:40:42.992634 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:42.992637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:42.992858 (+ 221us) server_negotiation.cc:407] Connection header received
0806 22:40:42.993398 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:42.993399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:42.993413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:42.993472 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:42.994126 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.994621 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.996238 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.996444 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.996505 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.999246 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:42.999248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:42.999248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:42.999257 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:43.000929 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:43.001387 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:43.001389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:43.001390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:43.001456 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:43.001731 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:43.001733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:43.001733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:43.001859 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:43.001934 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:43.002255 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 22:40:43.002360 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:40:43.002720 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:42.992506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:60699)
0806 22:40:42.992695 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:42.992734 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:42.993293 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:42.993555 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:42.993558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:42.993824 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:42.994039 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.994043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.994822 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.994823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.996139 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:42.996142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:42.996538 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:42.996540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:42.996692 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:42.997305 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:42.997317 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:42.999119 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:43.001024 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:43.001034 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:43.001036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:43.001298 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:43.001530 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:43.001531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:43.001531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:43.001652 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:43.001939 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:43.001947 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:43.002096 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 22:40:43.002220 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 22:40:43.002398 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":109,"threads_started":1}
I20260806 22:40:43.003304 8146 heartbeater.cc:347] Connected to a master server at 127.16.235.126:36595
I20260806 22:40:43.003443 8146 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:43.003701 8146 heartbeater.cc:511] Master 127.16.235.126:36595 requested a full tablet report, sending...
I20260806 22:40:43.004179 7680 ts_manager.cc:194] Registered new tserver with Master: 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747)
I20260806 22:40:43.004766 7680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:60699
W20260806 22:40:43.083923 8149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:40:43.084172 8149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:40:43.084201 8149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:40:43.085364 8149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:40:43.085403 8149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:40:43.085430 8149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:40:43.086970 8149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:43969
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=39181
--tserver_master_addrs=127.16.235.126:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:40:43.087180 8149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:40:43.087386 8149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:40:43.092692 8158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:43.092686 8159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:40:43.092690 8162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:40:43.092969 8149 server_base.cc:1034] running on GCE node
I20260806 22:40:43.093225 8149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:40:43.093513 8149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:40:43.094762 8149 hybrid_clock.cc:648] HybridClock initialized: now 1786056043094749 us; error 38 us; skew 500 ppm
Aug 06 22:40:43 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056043, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:40:43.097445 8149 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:40:43.097883 8149 webserver.cc:460] Webserver started at http://127.16.235.67:39181/ using document root <none> and password file <none>
I20260806 22:40:43.098076 8149 fs_manager.cc:359] Metadata directory not provided
I20260806 22:40:43.098131 8149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:40:43.099610 8149 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:40:43.100229 8176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:43.100421 8149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:40:43.100510 8149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "c1187f9ffd8d4f3aa007da32a81b78ed"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "a451037ac2ddb0d77e9d34caffa36107"
server_key_iv: "b9e03bf6f5241d93176ebe55c7097c9e"
server_key_version: "encryptionkey@0"
I20260806 22:40:43.100853 8149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:40:43.115432 8149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:40:43.117522 8149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:40:43.117647 8149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:40:43.117853 8149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:40:43.118186 8149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:40:43.118222 8149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:43.118245 8149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:40:43.118292 8149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:43.124393 8149 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:43969
I20260806 22:40:43.124498 8346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:43969 every 8 connection(s)
I20260806 22:40:43.124853 8149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:40:43 dist-test-slave-3kk6 krb5kdc[6810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056043, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:40:43.132597 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 8149
I20260806 22:40:43.135149 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.125688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:48959 (local address 127.16.235.126:36595)
0806 22:40:43.125766 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.125768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.125984 (+ 216us) server_negotiation.cc:407] Connection header received
0806 22:40:43.126453 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.126455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.126465 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.126518 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:43.127571 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.128148 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.129305 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.129498 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.129557 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.131895 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:43.131896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:43.131897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:43.131903 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:43.133661 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:43.134175 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:43.134176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:43.134176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:43.134239 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:43.134484 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:43.134485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:43.134485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:43.134614 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:43.134709 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:43.135013 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 22:40:43.135085 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:40:43.135406 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.125663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:48959)
0806 22:40:43.125851 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:43.125889 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:43.126366 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:43.126580 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:43.126584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:43.127257 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:40:43.127462 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.127468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.128225 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.128226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.129230 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.129233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.129572 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.129573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.129676 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.130206 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:40:43.130216 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:40:43.131802 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:40:43.133739 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:43.133747 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:43.133749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:43.134066 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:43.134307 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:40:43.134307 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:40:43.134308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:40:43.134415 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:40:43.134708 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:40:43.134714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:40:43.134851 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 22:40:43.134975 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 22:40:43.135144 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260806 22:40:43.135970 8347 heartbeater.cc:347] Connected to a master server at 127.16.235.126:36595
I20260806 22:40:43.136077 8347 heartbeater.cc:464] Registering TS with master...
I20260806 22:40:43.136256 8347 heartbeater.cc:511] Master 127.16.235.126:36595 requested a full tablet report, sending...
I20260806 22:40:43.136695 7680 ts_manager.cc:194] Registered new tserver with Master: c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
I20260806 22:40:43.137073 7680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:48959
I20260806 22:40:43.139803 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.133058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56138 (local address 127.16.235.126:36595)
0806 22:40:43.133221 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.133223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.133291 (+ 68us) server_negotiation.cc:407] Connection header received
0806 22:40:43.133360 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.133368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.133382 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.133454 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:43.134177 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.134861 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.135885 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.136185 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.136263 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.137612 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:43.137613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:43.137614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:43.137621 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:43.138915 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:43.139250 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:43.139251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:43.139251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:43.139303 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:43.139456 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:43.139456 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:43.139456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:43.139540 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:43.139650 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:43.139684 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:40:43.139744 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260806 22:40:43.140547 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:40:43.143155 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.140942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56146 (local address 127.16.235.126:36595)
0806 22:40:43.140980 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.140982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.141084 (+ 102us) server_negotiation.cc:407] Connection header received
0806 22:40:43.141144 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.141145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.141174 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.141231 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:43.141324 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.141763 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.142482 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.142713 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.142784 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.142905 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:43.142944 (+ 39us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 22:40:43.143100 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56146: token expired
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:40:43.143249 8351 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56146: token expired
W20260806 22:40:43.143466 7583 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.16.235.126:36595)
I20260806 22:40:43.149205 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.143757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56158 (local address 127.16.235.126:36595)
0806 22:40:43.143796 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.143798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.143806 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:40:43.143826 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.143826 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.143834 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.143889 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:40:43.144370 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.144810 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.145498 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.145684 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.145773 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.146916 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:40:43.146917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:40:43.146918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:40:43.146924 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:40:43.148069 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:43.148410 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:43.148411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:43.148412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:43.148476 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:40:43.148684 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:40:43.148684 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:40:43.148685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:40:43.148826 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:40:43.148935 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:43.149031 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:40:43.149109 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:40:43.150382 7680 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:40:43.152019 7680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:40:43.154260 7859 tablet_service.cc:1467] Processing CreateTablet for tablet b1b4a286ac54450dadec24274a9125c4 (DEFAULT_TABLE table=test-table [id=4b9dfdca73aa4ad094a1e58e96509e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:43.154498 7859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1b4a286ac54450dadec24274a9125c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:43.156224 7796 tablet_bootstrap.cc:492] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Bootstrap starting.
I20260806 22:40:43.156661 7796 tablet_bootstrap.cc:654] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:43.157472 7796 tablet_bootstrap.cc:492] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: No bootstrap required, opened a new log
I20260806 22:40:43.157527 7796 ts_tablet_manager.cc:1397] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:40:43.157675 7796 raft_consensus.cc:348] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:40:43.157783 7796 raft_consensus.cc:374] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:43.157843 7796 raft_consensus.cc:729] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a2571380b84f469c3c6c9328a7d30a, State: Initialized, Role: FOLLOWER
I20260806 22:40:43.157897 7796 consensus_queue.cc:260] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:40:43.158003 7796 ts_tablet_manager.cc:1428] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:40:43.158097 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.153927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:40747 (local address 127.0.0.1:43996)
0806 22:40:43.153977 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:43.153990 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:43.154029 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:43.154584 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:43.154586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:43.154588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:43.154631 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.154632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.155871 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.155872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.156825 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.156829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.157871 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.157873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.157934 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.157938 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:43.157991 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:40:43.158029 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:40:43.158419 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.154038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.16.235.66:40747)
0806 22:40:43.154282 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.154286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.154298 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:40:43.154340 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.154342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.154460 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.154561 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:43.154745 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.155791 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.156938 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.157760 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.157874 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.157941 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:43.158282 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 22:40:43.158324 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":130,"threads_started":1}
I20260806 22:40:43.158969 8053 tablet_service.cc:1467] Processing CreateTablet for tablet b1b4a286ac54450dadec24274a9125c4 (DEFAULT_TABLE table=test-table [id=4b9dfdca73aa4ad094a1e58e96509e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:43.159317 8053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1b4a286ac54450dadec24274a9125c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:43.159991 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.154223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55382 (local address 127.16.235.67:43969)
0806 22:40:43.154400 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.154404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.156410 (+ 2006us) server_negotiation.cc:407] Connection header received
0806 22:40:43.156461 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.156463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.156476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.156563 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:43.156708 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.157713 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.159016 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.159672 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.159753 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.159813 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:43.159901 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 22:40:43.159931 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260806 22:40:43.159999 8353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.153986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:43969 (local address 127.0.0.1:55382)
0806 22:40:43.156270 (+ 2284us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:43.156281 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:43.156337 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:43.156560 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:43.156561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:43.156562 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:43.156650 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.156652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.157803 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.157804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.158920 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.158924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.159754 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.159755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.159818 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.159822 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:40:43.159891 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:40:43.159929 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2265,"thread_start_us":107,"threads_started":1}
I20260806 22:40:43.160856 8249 tablet_service.cc:1467] Processing CreateTablet for tablet b1b4a286ac54450dadec24274a9125c4 (DEFAULT_TABLE table=test-table [id=4b9dfdca73aa4ad094a1e58e96509e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:40:43.161156 8249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1b4a286ac54450dadec24274a9125c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:40:43.161511 7998 tablet_bootstrap.cc:492] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: Bootstrap starting.
I20260806 22:40:43.162411 7998 tablet_bootstrap.cc:654] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:43.162781 7998 log.cc:824] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:43.163560 7998 tablet_bootstrap.cc:492] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: No bootstrap required, opened a new log
I20260806 22:40:43.163641 7998 ts_tablet_manager.cc:1397] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:40:43.164986 8359 tablet_bootstrap.cc:492] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: Bootstrap starting.
I20260806 22:40:43.165566 8359 tablet_bootstrap.cc:654] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: Neither blocks nor log segments found. Creating new log.
I20260806 22:40:43.165513 7998 raft_consensus.cc:348] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:40:43.165634 7998 raft_consensus.cc:374] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:43.165699 7998 raft_consensus.cc:729] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 208237e2644140fea5a3c19ae33db423, State: Initialized, Role: FOLLOWER
I20260806 22:40:43.165827 7998 consensus_queue.cc:260] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:40:43.166111 7998 ts_tablet_manager.cc:1428] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 22:40:43.166111 8146 heartbeater.cc:503] Master 127.16.235.126:36595 was elected leader, sending a full tablet report...
I20260806 22:40:43.166190 8359 log.cc:824] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: Log is configured to *not* fsync() on all Append() calls
I20260806 22:40:43.166981 8359 tablet_bootstrap.cc:492] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: No bootstrap required, opened a new log
I20260806 22:40:43.167038 8359 ts_tablet_manager.cc:1397] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:43.168385 8359 raft_consensus.cc:348] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:40:43.168529 8359 raft_consensus.cc:374] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:40:43.168551 8359 raft_consensus.cc:729] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1187f9ffd8d4f3aa007da32a81b78ed, State: Initialized, Role: FOLLOWER
I20260806 22:40:43.168686 8359 consensus_queue.cc:260] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:40:43.168968 8359 ts_tablet_manager.cc:1428] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:40:43.169008 8347 heartbeater.cc:503] Master 127.16.235.126:36595 was elected leader, sending a full tablet report...
W20260806 22:40:43.242396 8147 tablet.cc:2369] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:43.363159 7957 tablet.cc:2369] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:40:43.375490 8348 tablet.cc:2369] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:40:43.388819 7962 raft_consensus.cc:482] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:40:43.388993 7962 raft_consensus.cc:504] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:40:43.389375 7962 leader_election.cc:288] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
I20260806 22:40:43.392804 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.389751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:55653 (local address 127.16.235.66:40747)
0806 22:40:43.389813 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.389816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.389941 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:40:43.390057 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.390058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.390072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.390135 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:43.390325 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.390863 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.392059 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.392455 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.392533 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.392541 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:43.392700 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:40:43.392740 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:40:43.392794 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.389608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:43969 (local address 127.16.235.65:42799)
0806 22:40:43.389773 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:43.389788 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:43.389878 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:43.390065 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:43.390066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:43.390070 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:43.390138 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.390141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.390944 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.390945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.391819 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.391821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.392486 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.392487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.392561 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.392567 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:40:43.392621 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:40:43.392680 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":22,"thread_start_us":91,"threads_started":1}
I20260806 22:40:43.392779 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.389808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:42799 (local address 127.16.235.67:43969)
0806 22:40:43.389858 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.389860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.389873 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:40:43.389975 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.389976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.389987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.390046 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:40:43.390226 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.390848 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.391936 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.392386 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.392449 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.392457 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:43.392652 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 22:40:43.392700 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:40:43.392894 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.389617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:40747 (local address 127.16.235.65:55653)
0806 22:40:43.389840 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:43.389855 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:40:43.389957 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:40:43.390152 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:40:43.390152 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:40:43.390154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:40:43.390244 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.390246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.390966 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.390967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.391957 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:40:43.391960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.392545 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.392546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:40:43.392592 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.392595 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:40:43.392677 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:40:43.392718 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":98,"thread_start_us":86,"threads_started":1}
I20260806 22:40:43.393237 8280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b4a286ac54450dadec24274a9125c4" candidate_uuid: "25a2571380b84f469c3c6c9328a7d30a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" is_pre_election: true
I20260806 22:40:43.393445 8280 raft_consensus.cc:2454] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a2571380b84f469c3c6c9328a7d30a in term 0.
I20260806 22:40:43.393385 8085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b4a286ac54450dadec24274a9125c4" candidate_uuid: "25a2571380b84f469c3c6c9328a7d30a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "208237e2644140fea5a3c19ae33db423" is_pre_election: true
I20260806 22:40:43.393574 8085 raft_consensus.cc:2454] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a2571380b84f469c3c6c9328a7d30a in term 0.
I20260806 22:40:43.393718 7792 leader_election.cc:302] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25a2571380b84f469c3c6c9328a7d30a, c1187f9ffd8d4f3aa007da32a81b78ed; no voters:
I20260806 22:40:43.393895 7962 raft_consensus.cc:2781] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:40:43.393961 7962 raft_consensus.cc:482] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:40:43.393992 7962 raft_consensus.cc:3037] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:43.394840 7962 raft_consensus.cc:504] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:40:43.395002 7962 leader_election.cc:288] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 1 election: Requested vote from peers 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
I20260806 22:40:43.395157 8085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b4a286ac54450dadec24274a9125c4" candidate_uuid: "25a2571380b84f469c3c6c9328a7d30a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "208237e2644140fea5a3c19ae33db423"
I20260806 22:40:43.395242 8085 raft_consensus.cc:3037] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:43.395218 8280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b4a286ac54450dadec24274a9125c4" candidate_uuid: "25a2571380b84f469c3c6c9328a7d30a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed"
I20260806 22:40:43.395313 8280 raft_consensus.cc:3037] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:40:43.395949 8085 raft_consensus.cc:2454] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a2571380b84f469c3c6c9328a7d30a in term 1.
I20260806 22:40:43.396129 7789 leader_election.cc:302] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 208237e2644140fea5a3c19ae33db423, 25a2571380b84f469c3c6c9328a7d30a; no voters:
I20260806 22:40:43.396162 8280 raft_consensus.cc:2454] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a2571380b84f469c3c6c9328a7d30a in term 1.
I20260806 22:40:43.396237 7962 raft_consensus.cc:2781] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:40:43.396377 7962 raft_consensus.cc:686] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 LEADER]: Becoming Leader. State: Replica: 25a2571380b84f469c3c6c9328a7d30a, State: Running, Role: LEADER
I20260806 22:40:43.396464 7962 consensus_queue.cc:237] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:40:43.397289 7680 catalog_manager.cc:5495] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a reported cstate change: term changed from 0 to 1, leader changed from <none> to 25a2571380b84f469c3c6c9328a7d30a (127.16.235.65). New cstate: current_term: 1 leader_uuid: "25a2571380b84f469c3c6c9328a7d30a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } health_report { overall_health: HEALTHY } } }
I20260806 22:40:43.443933 8373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.440452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.16.235.65:40533)
0806 22:40:43.440660 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 22:40:43.440662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:40:43.440873 (+ 211us) server_negotiation.cc:407] Connection header received
0806 22:40:43.440993 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:40:43.440994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:40:43.441026 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:40:43.441110 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:40:43.441249 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.441780 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.442548 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:40:43.442849 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:40:43.442925 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:40:43.443081 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:40:43.443204 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:40:43.443288 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:40:43.443332 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:40:43.443354 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":105,"threads_started":1}
I20260806 22:40:43.445660 8280 raft_consensus.cc:1261] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [term 1 FOLLOWER]: Refusing update from remote peer 25a2571380b84f469c3c6c9328a7d30a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:43.445660 8085 raft_consensus.cc:1261] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [term 1 FOLLOWER]: Refusing update from remote peer 25a2571380b84f469c3c6c9328a7d30a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:40:43.445997 7962 consensus_queue.cc:1035] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:43.446204 8370 consensus_queue.cc:1035] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [LEADER]: Connected to new peer: Peer: permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:40:43.448936 8374 mvcc.cc:205] Tried to move back new op lower bound from 7315685553950736384 to 7315685553752784896. Current Snapshot: MvccSnapshot[applied={T|T < 7315685553950736384}]
I20260806 22:40:43.449035 8377 mvcc.cc:205] Tried to move back new op lower bound from 7315685553950736384 to 7315685553752784896. Current Snapshot: MvccSnapshot[applied={T|T < 7315685553950736384}]
I20260806 22:40:43.449908 8376 mvcc.cc:205] Tried to move back new op lower bound from 7315685553950736384 to 7315685553752784896. Current Snapshot: MvccSnapshot[applied={T|T < 7315685553950736384 or (T in {7315685553950736384})}]
I20260806 22:40:43.451100 7680 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:43.451210 7680 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:40:43.452028 7680 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Sending DeleteTablet for 3 replicas of tablet b1b4a286ac54450dadec24274a9125c4
I20260806 22:40:43.452401 7859 tablet_service.cc:1514] Processing DeleteTablet for tablet b1b4a286ac54450dadec24274a9125c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:43 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:46762
I20260806 22:40:43.452402 8249 tablet_service.cc:1514] Processing DeleteTablet for tablet b1b4a286ac54450dadec24274a9125c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:43 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:55382
I20260806 22:40:43.452461 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 7731
I20260806 22:40:43.453065 8053 tablet_service.cc:1514] Processing DeleteTablet for tablet b1b4a286ac54450dadec24274a9125c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:40:43 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:43996
I20260806 22:40:43.455094 8394 tablet_replica.cc:330] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: stopping tablet replica
I20260806 22:40:43.455253 8394 raft_consensus.cc:2229] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:43.455397 8394 raft_consensus.cc:2258] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:43.457466 8391 tablet_replica.cc:330] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: stopping tablet replica
I20260806 22:40:43.457623 8394 ts_tablet_manager.cc:1905] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:43.457650 8391 raft_consensus.cc:2229] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:40:43.457806 8391 raft_consensus.cc:2258] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:40:43.458572 8391 ts_tablet_manager.cc:1905] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:40:43.459739 8394 ts_tablet_manager.cc:1918] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:43.459873 8394 log.cc:1196] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/b1b4a286ac54450dadec24274a9125c4
I20260806 22:40:43.460253 8394 ts_tablet_manager.cc:1939] T b1b4a286ac54450dadec24274a9125c4 P 208237e2644140fea5a3c19ae33db423: Deleting consensus metadata
I20260806 22:40:43.460460 8391 ts_tablet_manager.cc:1918] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:40:43.460554 8391 log.cc:1196] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/b1b4a286ac54450dadec24274a9125c4
I20260806 22:40:43.460995 7662 catalog_manager.cc:4841] TS 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747): tablet b1b4a286ac54450dadec24274a9125c4 (table test-table [id=4b9dfdca73aa4ad094a1e58e96509e4f]) successfully deleted
I20260806 22:40:43.461045 8391 ts_tablet_manager.cc:1939] T b1b4a286ac54450dadec24274a9125c4 P c1187f9ffd8d4f3aa007da32a81b78ed: Deleting consensus metadata
I20260806 22:40:43.461547 7665 catalog_manager.cc:4841] TS c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969): tablet b1b4a286ac54450dadec24274a9125c4 (table test-table [id=4b9dfdca73aa4ad094a1e58e96509e4f]) successfully deleted
W20260806 22:40:43.472503 7664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:40533 (error 108)
I20260806 22:40:43.472671 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 7961
I20260806 22:40:43.489399 8353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:40:43.473073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:40533 (local address 127.0.0.1:46772)
0806 22:40:43.489197 (+ 16124us) negotiation.cc:107] Waiting for socket to connect
0806 22:40:43.489327 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16079}
W20260806 22:40:43.494571 7664 catalog_manager.cc:4584] TS 25a2571380b84f469c3c6c9328a7d30a (127.16.235.65:40533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1b4a286ac54450dadec24274a9125c4: Network error: Client connection negotiation failed: client connection to 127.16.235.65:40533: connect: Connection refused (error 111)
I20260806 22:40:43.499828 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 8149
I20260806 22:40:43.511535 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 7633
I20260806 22:41:03.521507 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36595
--webserver_interface=127.16.235.126
--webserver_port=39287
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:36595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:03.613373 8398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:03.613636 8398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:03.613663 8398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:03.614974 8398 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:41:03.615036 8398 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:41:03.615049 8398 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:41:03.615062 8398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:03.615077 8398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:03.615090 8398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:03.615152 8398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:03.615171 8398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:03.616909 8398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:36595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=39287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:03.617219 8398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:03.617496 8398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:41:03.623303 8398 server_base.cc:1034] running on GCE node
W20260806 22:41:03.623227 8412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:03.623246 8405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:03.623256 8408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:03.623764 8398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:03.624073 8398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:03.625247 8398 hybrid_clock.cc:648] HybridClock initialized: now 1786056063625224 us; error 42 us; skew 500 ppm
Aug 06 22:41:03 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:03.628258 8398 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:03.628788 8398 webserver.cc:460] Webserver started at http://127.16.235.126:39287/ using document root <none> and password file <none>
I20260806 22:41:03.629032 8398 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:03.629078 8398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:03.630554 8398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:03.631414 8425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:03.631621 8398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:03.631737 8398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "ac6f179d64c946e787909eab8ceb2a4a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "24795d75e0220704df21ce5456d0f6f2"
server_key_iv: "0d80d97798136f2709508e35eeb1649b"
server_key_version: "encryptionkey@0"
I20260806 22:41:03.632138 8398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:03.643080 8398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:03.645467 8398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:03.645604 8398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:03.650130 8398 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:36595
I20260806 22:41:03.650224 8505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:36595 every 8 connection(s)
I20260806 22:41:03.650704 8398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:03.652436 8507 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:41:03.653216 8507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Bootstrap starting.
I20260806 22:41:03.659353 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 8398
I20260806 22:41:03.659765 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:40533
--local_ip_for_outbound_sockets=127.16.235.65
--tserver_master_addrs=127.16.235.126:36595
--webserver_port=43889
--webserver_interface=127.16.235.65
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:41:03.663959 8507 log.cc:824] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:03.666476 8507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:41:03.666906 8507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Bootstrap complete.
I20260806 22:41:03.669416 8507 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:41:03.669687 8507 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6f179d64c946e787909eab8ceb2a4a, State: Initialized, Role: FOLLOWER
I20260806 22:41:03.669879 8507 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:41:03.669961 8507 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:03.670028 8507 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:03.670084 8507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:41:03.671386 8507 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:41:03.671494 8507 leader_election.cc:302] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6f179d64c946e787909eab8ceb2a4a; no voters:
I20260806 22:41:03.671782 8507 leader_election.cc:288] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:41:03.671824 8512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:41:03.671957 8512 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [term 3 LEADER]: Becoming Leader. State: Replica: ac6f179d64c946e787909eab8ceb2a4a, State: Running, Role: LEADER
I20260806 22:41:03.672128 8512 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } }
I20260806 22:41:03.672137 8507 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:03.673468 8514 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ac6f179d64c946e787909eab8ceb2a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } } }
I20260806 22:41:03.673468 8515 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6f179d64c946e787909eab8ceb2a4a. Latest consensus state: current_term: 3 leader_uuid: "ac6f179d64c946e787909eab8ceb2a4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f179d64c946e787909eab8ceb2a4a" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36595 } } }
I20260806 22:41:03.673592 8514 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:03.673766 8515 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:03.673908 8531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:03.675006 8531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:03.675148 8531 catalog_manager.cc:1259] Loaded cluster ID: e42c8b2dc8784192aa6ae51aea75ade1
I20260806 22:41:03.675220 8531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:03.676105 8531 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:03.676328 8531 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Loaded TSK: 0
I20260806 22:41:03.676390 8531 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ac6f179d64c946e787909eab8ceb2a4a: Loaded TSK: 1
F20260806 22:41:03.676642 8531 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f74a7d439ad google::LogMessage::Fail() at ??:0
@ 0x7f74a7d4688a google::LogMessage::SendToLog() at ??:0
@ 0x7f74a7d434de google::LogMessage::Flush() at ??:0
@ 0x7f74a7d44a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f74aa4b0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f74aa4add41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f74afca11cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f74afca148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f74afca1594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f74afc67c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f74afc71ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f74a9c110d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f74a9c06954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f74aaa6a6db start_thread at ??:0
@ 0x7f74a684161f clone at ??:0
W20260806 22:41:03.774106 8509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:03.774354 8509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:03.774384 8509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:03.775707 8509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:03.775759 8509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:03.775789 8509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:41:03.777352 8509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:40533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=43889
--tserver_master_addrs=127.16.235.126:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:03.777637 8509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:03.777907 8509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:03.783433 8569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:03.783494 8568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:03.783433 8573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:03.783944 8509 server_base.cc:1034] running on GCE node
I20260806 22:41:03.784183 8509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:03.784534 8509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:03.785730 8509 hybrid_clock.cc:648] HybridClock initialized: now 1786056063785712 us; error 37 us; skew 500 ppm
Aug 06 22:41:03 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:03.789434 8509 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:41:03.790114 8509 webserver.cc:460] Webserver started at http://127.16.235.65:43889/ using document root <none> and password file <none>
I20260806 22:41:03.790446 8509 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:03.790535 8509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:03.792645 8509 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:41:03.793583 8588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:03.793813 8509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:03.793922 8509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "25a2571380b84f469c3c6c9328a7d30a"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "b119b94ff7268f99c59d3321022c7e35"
server_key_iv: "a3c2a6461020a1bd7f0eea393b9ae1b8"
server_key_version: "encryptionkey@0"
I20260806 22:41:03.794425 8509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:03.809016 8509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:03.812265 8509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:03.812458 8509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:03.812783 8509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:03.813460 8596 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:41:03.815249 8509 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:41:03.815322 8509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:41:03.815361 8509 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:41:03.816318 8509 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:41:03.816380 8509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:41:03.816495 8596 tablet_bootstrap.cc:492] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Bootstrap starting.
I20260806 22:41:03.824645 8509 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:40533
I20260806 22:41:03.824811 8760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:40533 every 8 connection(s)
I20260806 22:41:03.825249 8509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:41:03.827543 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 8509
I20260806 22:41:03.827903 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:40747
--local_ip_for_outbound_sockets=127.16.235.66
--tserver_master_addrs=127.16.235.126:36595
--webserver_port=43879
--webserver_interface=127.16.235.66
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:41:03.829324 8596 log.cc:824] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:03.832692 8596 tablet_bootstrap.cc:492] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:41:03.833132 8596 tablet_bootstrap.cc:492] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Bootstrap complete.
I20260806 22:41:03.834679 8596 ts_tablet_manager.cc:1397] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.011s
I20260806 22:41:03.836719 8596 raft_consensus.cc:348] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:41:03.837213 8596 raft_consensus.cc:729] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a2571380b84f469c3c6c9328a7d30a, State: Initialized, Role: FOLLOWER
I20260806 22:41:03.837423 8596 consensus_queue.cc:260] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:41:03.837823 8596 ts_tablet_manager.cc:1428] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 22:41:03.900728 8769 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:03.826043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:42305)
0806 22:41:03.826265 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:03.826320 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:03.827013 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:03.900467 (+ 73454us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":89,"threads_started":1}
W20260806 22:41:03.901057 8763 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:36595 (0 consecutive failures): Network error: Failed to ping master at 127.16.235.126:36595: Client connection negotiation failed: client connection to 127.16.235.126:36595: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:41:03.901475 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:03.901341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:38007)
0806 22:41:03.901397 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:03.901447 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:41:03.901836 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:03.901748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:51175)
0806 22:41:03.901794 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:03.901821 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:41:03.901975 8763 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:41:03.921995 8771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:03.922225 8771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:03.922251 8771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:03.923520 8771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:03.923568 8771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:03.923595 8771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:41:03.925105 8771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:40747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=43879
--tserver_master_addrs=127.16.235.126:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:03.925356 8771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:03.925611 8771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:03.931075 8783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:03.931116 8771 server_base.cc:1034] running on GCE node
W20260806 22:41:03.931121 8781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:03.931082 8780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:03.931680 8771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:03.931957 8771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:03.933173 8771 hybrid_clock.cc:648] HybridClock initialized: now 1786056063933157 us; error 43 us; skew 500 ppm
Aug 06 22:41:03 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:03.936103 8771 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:41:03.936580 8771 webserver.cc:460] Webserver started at http://127.16.235.66:43879/ using document root <none> and password file <none>
I20260806 22:41:03.936839 8771 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:03.936913 8771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:03.938295 8771 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:03.938980 8792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:03.939162 8771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:03.939280 8771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "208237e2644140fea5a3c19ae33db423"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "6a362bc72c3e88e8030e1c98bac1ed17"
server_key_iv: "d8eb34e7285475ccb40d1ae151301c82"
server_key_version: "encryptionkey@0"
I20260806 22:41:03.939700 8771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:03.970330 8771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:03.972820 8771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:03.973013 8771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:03.973278 8771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:03.973613 8771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:03.973649 8771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:03.973675 8771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:03.973690 8771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:03.980499 8771 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:40747
I20260806 22:41:03.980546 8954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:40747 every 8 connection(s)
I20260806 22:41:03.980943 8771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:41:03.982089 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:03.981663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:59739)
0806 22:41:03.981837 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:03.981926 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
W20260806 22:41:03.982519 8955 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:36595 (0 consecutive failures): Network error: Failed to ping master at 127.16.235.126:36595: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
I20260806 22:41:03.982887 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:03.982768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:37583)
0806 22:41:03.982826 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:03.982862 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:41:03.983228 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:03.983129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:45125)
0806 22:41:03.983178 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:03.983204 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:41:03.983356 8955 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:41:03.986109 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 8771
I20260806 22:41:03.986369 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:43969
--local_ip_for_outbound_sockets=127.16.235.67
--tserver_master_addrs=127.16.235.126:36595
--webserver_port=39181
--webserver_interface=127.16.235.67
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:04.073119 8962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:04.073390 8962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:04.073433 8962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:04.074641 8962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:04.074757 8962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:04.074806 8962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:41:04.076550 8962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:43969
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=39181
--tserver_master_addrs=127.16.235.126:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:04.076869 8962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:04.077136 8962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:04.082568 8971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:04.082633 8974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:04.082597 8970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:04.082880 8962 server_base.cc:1034] running on GCE node
I20260806 22:41:04.083089 8962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:04.083307 8962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:04.084477 8962 hybrid_clock.cc:648] HybridClock initialized: now 1786056064084464 us; error 29 us; skew 500 ppm
Aug 06 22:41:04 dist-test-slave-3kk6 krb5kdc[6810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056064, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:04.087249 8962 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:41:04.087671 8962 webserver.cc:460] Webserver started at http://127.16.235.67:39181/ using document root <none> and password file <none>
I20260806 22:41:04.087895 8962 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:04.087941 8962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:04.089272 8962 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:41:04.089893 8984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:04.090090 8962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:04.090193 8962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "c1187f9ffd8d4f3aa007da32a81b78ed"
format_stamp: "Formatted at 2026-08-06 22:40:21 on dist-test-slave-3kk6"
server_key: "a451037ac2ddb0d77e9d34caffa36107"
server_key_iv: "b9e03bf6f5241d93176ebe55c7097c9e"
server_key_version: "encryptionkey@0"
I20260806 22:41:04.090574 8962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:04.105930 8962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:04.108260 8962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:04.108403 8962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:04.108614 8962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:04.108990 8962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:04.109030 8962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:04.109086 8962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:04.109119 8962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:04.115777 8962 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:43969
I20260806 22:41:04.115914 9163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:43969 every 8 connection(s)
I20260806 22:41:04.116267 8962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:41:04.117477 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:04.117056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:46101)
0806 22:41:04.117241 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:04.117329 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
W20260806 22:41:04.117906 9164 heartbeater.cc:644] Failed to heartbeat to 127.16.235.126:36595 (0 consecutive failures): Network error: Failed to ping master at 127.16.235.126:36595: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
I20260806 22:41:04.118218 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:04.118108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:60015)
0806 22:41:04.118160 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:04.118188 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:41:04.118503 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:04.118417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:37709)
0806 22:41:04.118461 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:04.118484 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:41:04.118619 9164 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:41:04.123158 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 8962
I20260806 22:41:04.902828 9170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:04.902517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:50663)
0806 22:41:04.902701 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:04.902754 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260806 22:41:04.984025 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:04.983744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:59309)
0806 22:41:04.983896 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:04.983948 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260806 22:41:05.077840 9177 raft_consensus.cc:482] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:05.078030 9177 raft_consensus.cc:504] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:41:05.078423 9177 leader_election.cc:288] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
Aug 06 22:41:05 dist-test-slave-3kk6 krb5kdc[6810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.66@KRBTEST.COM
Aug 06 22:41:05 dist-test-slave-3kk6 krb5kdc[6810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.67@KRBTEST.COM
I20260806 22:41:05.090416 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:05.078724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:43969 (local address 127.16.235.65:38753)
0806 22:41:05.078862 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:05.078871 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:05.078930 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:05.079943 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:05.079944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:05.080314 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:05.080534 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:05.080541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:05.081709 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:05.081710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:05.082530 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:05.082533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:05.083193 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:05.083194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:05.083314 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:05.084025 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:05.084033 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:05.085992 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:05.088329 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:05.088330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:05.088330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:05.088632 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:05.088993 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:05.088994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:05.088994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:05.089164 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:05.089751 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:05.089759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:05.089930 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 22:41:05.090056 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 22:41:05.090283 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":58,"thread_start_us":56,"threads_started":1}
I20260806 22:41:05.090433 9183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:05.078811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:38753 (local address 127.16.235.67:43969)
0806 22:41:05.078981 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:05.078988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:05.079013 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:41:05.079568 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:05.079572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:05.079652 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:05.079816 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:05.080673 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:05.081589 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:05.082643 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:05.083082 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:05.083209 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:05.086146 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:05.086150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:05.086151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:05.086171 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:05.088254 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:05.088753 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:05.088756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:05.088756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:05.088899 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:05.089259 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:05.089260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:05.089261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:05.089660 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:05.089772 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:05.090147 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 22:41:05.090318 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260806 22:41:05.090458 9181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:05.078798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:58875 (local address 127.16.235.66:40747)
0806 22:41:05.078973 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:05.078987 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:05.079018 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:41:05.079609 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:05.079612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:05.079652 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:05.079816 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:05.080679 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:05.081571 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:05.082634 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:05.083081 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:05.083209 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:05.085671 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:05.085676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:05.085676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:05.085690 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:05.088045 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:05.088751 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:05.088756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:05.088756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:05.088899 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:05.089259 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:05.089260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:05.089261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:05.089660 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:05.089772 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:05.090147 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 22:41:05.090317 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":57,"threads_started":1}
I20260806 22:41:05.090472 9170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:05.078649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:40747 (local address 127.16.235.65:58875)
0806 22:41:05.078735 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:05.078751 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:05.078818 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:05.079911 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:05.079914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:05.080318 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:05.080534 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:05.080541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:05.081688 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:05.081689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:05.082529 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:05.082532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:05.083186 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:05.083187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:05.083313 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:05.084093 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:05.084098 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:05.085553 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:05.088155 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:05.088162 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:05.088164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:05.088614 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:05.088993 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:05.088994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:05.088994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:05.089164 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:05.089751 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:05.089759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:05.089930 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 22:41:05.090056 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 22:41:05.090351 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":171}
W20260806 22:41:05.091904 8592 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
W20260806 22:41:05.092022 8590 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
I20260806 22:41:05.092079 8590 leader_election.cc:302] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25a2571380b84f469c3c6c9328a7d30a; no voters: 208237e2644140fea5a3c19ae33db423, c1187f9ffd8d4f3aa007da32a81b78ed
I20260806 22:41:05.092249 9177 raft_consensus.cc:2726] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:41:05.119249 9184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:05.118979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:45099)
0806 22:41:05.119145 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:05.119203 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260806 22:41:05.903730 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:05.903427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:59641)
0806 22:41:05.903587 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:05.903646 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260806 22:41:05.984833 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:05.984545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:36631)
0806 22:41:05.984730 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:05.984783 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":74,"threads_started":1}
I20260806 22:41:06.120087 9189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:06.119835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:45617)
0806 22:41:06.119990 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:06.120041 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260806 22:41:06.904609 9191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:06.904322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:47647)
0806 22:41:06.904500 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:06.904555 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":80,"threads_started":1}
I20260806 22:41:06.985630 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:06.985400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:41311)
0806 22:41:06.985536 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:06.985584 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260806 22:41:07.120850 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:07.120596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:42991)
0806 22:41:07.120771 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:07.120804 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260806 22:41:07.149996 9195 raft_consensus.cc:482] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:07.150113 9195 raft_consensus.cc:504] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:41:07.150375 9195 leader_election.cc:288] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
W20260806 22:41:07.150897 8592 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
W20260806 22:41:07.150981 8590 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
I20260806 22:41:07.151024 8590 leader_election.cc:302] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25a2571380b84f469c3c6c9328a7d30a; no voters: 208237e2644140fea5a3c19ae33db423, c1187f9ffd8d4f3aa007da32a81b78ed
I20260806 22:41:07.151126 9195 raft_consensus.cc:2726] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:41:07.905462 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:07.905180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:42579)
0806 22:41:07.905357 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:07.905412 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":79,"threads_started":1}
I20260806 22:41:07.986337 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:07.986092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:56969)
0806 22:41:07.986255 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:07.986294 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
I20260806 22:41:08.121518 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:08.121299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:42713)
0806 22:41:08.121441 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:08.121474 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260806 22:41:08.906402 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:08.906146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:56377)
0806 22:41:08.906305 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:08.906348 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260806 22:41:08.987248 9200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:08.986990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:54637)
0806 22:41:08.987160 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:08.987202 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
I20260806 22:41:09.122275 9204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:09.121997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:42163)
0806 22:41:09.122171 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:09.122219 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260806 22:41:09.521899 9205 raft_consensus.cc:482] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:09.522022 9205 raft_consensus.cc:504] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:41:09.522217 9205 leader_election.cc:288] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
W20260806 22:41:09.522769 8592 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
W20260806 22:41:09.522864 8590 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
I20260806 22:41:09.522902 8590 leader_election.cc:302] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25a2571380b84f469c3c6c9328a7d30a; no voters: 208237e2644140fea5a3c19ae33db423, c1187f9ffd8d4f3aa007da32a81b78ed
I20260806 22:41:09.523001 9205 raft_consensus.cc:2726] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:41:09.907269 9206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:09.906988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:52605)
0806 22:41:09.907162 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:09.907217 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":83,"threads_started":1}
I20260806 22:41:09.988071 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:09.987795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:42929)
0806 22:41:09.987974 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:09.988023 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260806 22:41:10.123082 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:10.122838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:50193)
0806 22:41:10.122993 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:10.123033 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260806 22:41:10.908160 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:10.907908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:40505)
0806 22:41:10.908056 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:10.908109 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260806 22:41:10.988823 9210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:10.988585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:54567)
0806 22:41:10.988726 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:10.988774 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260806 22:41:11.123905 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:11.123634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:57739)
0806 22:41:11.123810 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:11.123854 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":104,"threads_started":1}
I20260806 22:41:11.908953 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:11.908678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:54881)
0806 22:41:11.908859 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:11.908902 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":77,"threads_started":1}
I20260806 22:41:11.989601 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:11.989352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:43801)
0806 22:41:11.989497 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:11.989544 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260806 22:41:12.124732 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:12.124486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:42807)
0806 22:41:12.124639 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:12.124683 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260806 22:41:12.484308 9220 raft_consensus.cc:482] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:12.484457 9220 raft_consensus.cc:504] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:41:12.484670 9220 leader_election.cc:288] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
W20260806 22:41:12.485070 8590 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
W20260806 22:41:12.485123 8592 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
I20260806 22:41:12.485152 8590 leader_election.cc:302] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25a2571380b84f469c3c6c9328a7d30a; no voters: 208237e2644140fea5a3c19ae33db423, c1187f9ffd8d4f3aa007da32a81b78ed
I20260806 22:41:12.485263 9220 raft_consensus.cc:2726] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:41:12.909775 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:12.909517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:48121)
0806 22:41:12.909676 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:12.909722 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260806 22:41:12.990528 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:12.990201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:34483)
0806 22:41:12.990397 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:12.990454 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":112,"threads_started":1}
I20260806 22:41:13.125582 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:13.125291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:55329)
0806 22:41:13.125481 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:13.125532 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":103,"threads_started":1}
I20260806 22:41:13.910734 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:13.910371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:35519)
0806 22:41:13.910545 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:13.910595 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260806 22:41:13.991452 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:13.991154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:54943)
0806 22:41:13.991323 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:13.991382 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260806 22:41:14.072649 9230 raft_consensus.cc:482] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:14.072777 9230 raft_consensus.cc:504] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:41:14.073030 9230 leader_election.cc:288] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
W20260806 22:41:14.073503 8590 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
W20260806 22:41:14.073596 8592 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
I20260806 22:41:14.073691 8592 leader_election.cc:302] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25a2571380b84f469c3c6c9328a7d30a; no voters: 208237e2644140fea5a3c19ae33db423, c1187f9ffd8d4f3aa007da32a81b78ed
I20260806 22:41:14.073786 9230 raft_consensus.cc:2726] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:41:14.126338 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:14.126099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:54683)
0806 22:41:14.126254 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:14.126289 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260806 22:41:14.911633 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:14.911341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:57291)
0806 22:41:14.911524 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:14.911581 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":80,"threads_started":1}
I20260806 22:41:14.992352 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:14.992095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:60251)
0806 22:41:14.992250 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:14.992299 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260806 22:41:15.127257 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:15.126995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:42777)
0806 22:41:15.127147 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:15.127203 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260806 22:41:15.912523 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:15.912259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:48735)
0806 22:41:15.912415 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:15.912469 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260806 22:41:15.993292 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:15.992894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:41439)
0806 22:41:15.993052 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:15.993095 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260806 22:41:16.128211 9240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:16.127900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:46927)
0806 22:41:16.128100 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:16.128157 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":82,"threads_started":1}
I20260806 22:41:16.913406 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:16.913144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:56575)
0806 22:41:16.913297 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:16.913348 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260806 22:41:16.994284 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:16.993988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:57345)
0806 22:41:16.994164 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:16.994232 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
I20260806 22:41:17.129189 9243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:17.128829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:45555)
0806 22:41:17.129081 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:17.129137 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":75,"threads_started":1}
I20260806 22:41:17.914350 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:17.914085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:55533)
0806 22:41:17.914243 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:17.914297 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260806 22:41:17.995332 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:17.995067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:54403)
0806 22:41:17.995222 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:17.995280 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260806 22:41:18.130033 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:18.129770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:41525)
0806 22:41:18.129934 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:18.129980 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260806 22:41:18.828059 9248 raft_consensus.cc:482] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:18.828156 9248 raft_consensus.cc:504] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "208237e2644140fea5a3c19ae33db423" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 40747 } } peers { permanent_uuid: "c1187f9ffd8d4f3aa007da32a81b78ed" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43969 } } peers { permanent_uuid: "25a2571380b84f469c3c6c9328a7d30a" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 40533 } }
I20260806 22:41:18.828342 9248 leader_election.cc:288] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747), c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969)
W20260806 22:41:18.828894 8590 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 208237e2644140fea5a3c19ae33db423 (127.16.235.66:40747): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
W20260806 22:41:18.828999 8592 leader_election.cc:341] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1187f9ffd8d4f3aa007da32a81b78ed (127.16.235.67:43969): Not found: Tablet not found: b1b4a286ac54450dadec24274a9125c4
I20260806 22:41:18.829077 8592 leader_election.cc:302] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25a2571380b84f469c3c6c9328a7d30a; no voters: 208237e2644140fea5a3c19ae33db423, c1187f9ffd8d4f3aa007da32a81b78ed
I20260806 22:41:18.829149 9248 raft_consensus.cc:2726] T b1b4a286ac54450dadec24274a9125c4 P 25a2571380b84f469c3c6c9328a7d30a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:41:18.915382 9251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:18.915113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.65:51497)
0806 22:41:18.915282 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:18.915332 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260806 22:41:18.996250 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:18.995987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.66:48915)
0806 22:41:18.996144 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:18.996188 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260806 22:41:19.130312 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 8509
I20260806 22:41:19.137238 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:19.135672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:36595 (local address 127.16.235.67:58041)
0806 22:41:19.137098 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:19.137163 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.126:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1395,"thread_start_us":66,"threads_started":1}
I20260806 22:41:19.143954 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 8771
I20260806 22:41:19.155187 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 8962
2026-08-06T22:41:19Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59222 ms)
[----------] 30 tests from SecurityITest (243151 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:19 dist-test-slave-3kk6 krb5kdc[9259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:19 dist-test-slave-3kk6 krb5kdc[9259](info): set up 2 sockets
Aug 06 22:41:19 dist-test-slave-3kk6 krb5kdc[9259](info): commencing operation
krb5kdc: starting...
W20260806 22:41:20.397600 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:20 dist-test-slave-3kk6 krb5kdc[9259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:20.533434 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38157
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:38157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:20.621906 9281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:20.622201 9281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:20.622244 9281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:20.623510 9281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:20.623600 9281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:20.623624 9281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:20.623646 9281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:20.623677 9281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:20.625296 9281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:38157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:20.625630 9281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:20.625907 9281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:20.631422 9289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:20.631516 9290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:20.631620 9281 server_base.cc:1034] running on GCE node
W20260806 22:41:20.631517 9293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:20.632040 9281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:20.632373 9281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:20.633564 9281 hybrid_clock.cc:648] HybridClock initialized: now 1786056080633555 us; error 45 us; skew 500 ppm
Aug 06 22:41:20 dist-test-slave-3kk6 krb5kdc[9259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056080, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:20.639657 9281 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:20.640183 9281 webserver.cc:460] Webserver started at http://127.16.235.126:44729/ using document root <none> and password file <none>
I20260806 22:41:20.640466 9281 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:20.640543 9281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:20.640769 9281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:20.641978 9281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "da9bded7485c4bd28a6f48403cab5fa9"
format_stamp: "Formatted at 2026-08-06 22:41:20 on dist-test-slave-3kk6"
server_key: "0fb285f1ba9c734642e13a80ca128442"
server_key_iv: "1bf04fab11f2d1408de399f73533507a"
server_key_version: "encryptionkey@0"
I20260806 22:41:20.642413 9281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "da9bded7485c4bd28a6f48403cab5fa9"
format_stamp: "Formatted at 2026-08-06 22:41:20 on dist-test-slave-3kk6"
server_key: "0fb285f1ba9c734642e13a80ca128442"
server_key_iv: "1bf04fab11f2d1408de399f73533507a"
server_key_version: "encryptionkey@0"
I20260806 22:41:20.644140 9281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:41:20.645008 9302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:20.645195 9281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:20.645313 9281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "da9bded7485c4bd28a6f48403cab5fa9"
format_stamp: "Formatted at 2026-08-06 22:41:20 on dist-test-slave-3kk6"
server_key: "0fb285f1ba9c734642e13a80ca128442"
server_key_iv: "1bf04fab11f2d1408de399f73533507a"
server_key_version: "encryptionkey@0"
I20260806 22:41:20.645395 9281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:20.672554 9281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:20.675240 9281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:20.675448 9281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:20.679941 9281 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:38157
I20260806 22:41:20.679973 9370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:38157 every 8 connection(s)
I20260806 22:41:20.680439 9281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:20.681422 9371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:20.683857 9371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9: Bootstrap starting.
I20260806 22:41:20.684514 9371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:20.684834 9371 log.cc:824] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:20.685577 9371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9: No bootstrap required, opened a new log
I20260806 22:41:20.687362 9371 raft_consensus.cc:348] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9bded7485c4bd28a6f48403cab5fa9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38157 } }
I20260806 22:41:20.687515 9371 raft_consensus.cc:374] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:20.687538 9371 raft_consensus.cc:729] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9bded7485c4bd28a6f48403cab5fa9, State: Initialized, Role: FOLLOWER
I20260806 22:41:20.687728 9371 consensus_queue.cc:260] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9bded7485c4bd28a6f48403cab5fa9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38157 } }
I20260806 22:41:20.687825 9371 raft_consensus.cc:388] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:20.687883 9371 raft_consensus.cc:482] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:20.687934 9371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:20.688566 9371 raft_consensus.cc:504] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9bded7485c4bd28a6f48403cab5fa9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38157 } }
I20260806 22:41:20.688692 9371 leader_election.cc:302] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da9bded7485c4bd28a6f48403cab5fa9; no voters:
I20260806 22:41:20.688898 9371 leader_election.cc:288] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:20.688964 9378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:20.689170 9378 raft_consensus.cc:686] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [term 1 LEADER]: Becoming Leader. State: Replica: da9bded7485c4bd28a6f48403cab5fa9, State: Running, Role: LEADER
I20260806 22:41:20.689184 9371 sys_catalog.cc:564] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:20.689289 9378 consensus_queue.cc:237] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9bded7485c4bd28a6f48403cab5fa9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38157 } }
I20260806 22:41:20.689715 9380 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da9bded7485c4bd28a6f48403cab5fa9. Latest consensus state: current_term: 1 leader_uuid: "da9bded7485c4bd28a6f48403cab5fa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9bded7485c4bd28a6f48403cab5fa9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38157 } } }
I20260806 22:41:20.689862 9379 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da9bded7485c4bd28a6f48403cab5fa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da9bded7485c4bd28a6f48403cab5fa9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38157 } } }
I20260806 22:41:20.689884 9380 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:20.689965 9379 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:20.690353 9387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:20.690506 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 9281
I20260806 22:41:20.690624 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:20.690873 17325 external_mini_cluster.cc:1428] Setting key 2598afdb90b6596c68cb10aae038ae68
I20260806 22:41:20.691035 9387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:20.693043 9387 catalog_manager.cc:1347] Generated new cluster ID: 3f6bf532d64c4c49b5d588df5f6e8cff
I20260806 22:41:20.693106 9387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:41:20 dist-test-slave-3kk6 krb5kdc[9259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:20.703671 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:20.691521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60634 (local address 127.16.235.126:38157)
0806 22:41:20.691793 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:20.691803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:20.691837 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:41:20.691972 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:20.691984 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:20.692045 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:20.692199 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:20.693005 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:20.694444 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:20.695635 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:20.696280 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:20.696396 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:20.699576 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:20.699581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:20.699581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:20.699596 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:20.701789 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:20.702235 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:20.702238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:20.702239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:20.702370 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:20.702589 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:20.702590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:20.702590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:20.702981 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:20.703163 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:20.703238 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:41:20.703406 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":92,"threads_started":1}
I20260806 22:41:20.709643 9387 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:20.710429 9387 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:20.722585 9387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9: Generated new TSK 0
I20260806 22:41:20.722930 9387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:41:20.753934 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38157
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:20.842532 9409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:20.842839 9409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:20.842869 9409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:20.844071 9409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:20.844121 9409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:20.844148 9409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:41:20.845677 9409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:20.845942 9409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:20.846197 9409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:20.851876 9421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:20.851882 9419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:20.851892 9418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:20.851958 9409 server_base.cc:1034] running on GCE node
I20260806 22:41:20.852386 9409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:20.852669 9409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:20.853847 9409 hybrid_clock.cc:648] HybridClock initialized: now 1786056080853825 us; error 33 us; skew 500 ppm
Aug 06 22:41:20 dist-test-slave-3kk6 krb5kdc[9259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056080, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:20.856606 9409 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:41:20.857105 9409 webserver.cc:460] Webserver started at http://127.16.235.65:37029/ using document root <none> and password file <none>
I20260806 22:41:20.857349 9409 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:20.857422 9409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:20.857586 9409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:20.858786 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "648e4ac8d33548eb9048e4772a8a47b5"
format_stamp: "Formatted at 2026-08-06 22:41:20 on dist-test-slave-3kk6"
server_key: "0dd97605f2d1524a1bc261be9bd11b51"
server_key_iv: "b369b6822e948180ad1636f610dca886"
server_key_version: "encryptionkey@0"
I20260806 22:41:20.859182 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "648e4ac8d33548eb9048e4772a8a47b5"
format_stamp: "Formatted at 2026-08-06 22:41:20 on dist-test-slave-3kk6"
server_key: "0dd97605f2d1524a1bc261be9bd11b51"
server_key_iv: "b369b6822e948180ad1636f610dca886"
server_key_version: "encryptionkey@0"
I20260806 22:41:20.860672 9409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:41:20.861450 9431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:20.861631 9409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:41:20.861743 9409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "648e4ac8d33548eb9048e4772a8a47b5"
format_stamp: "Formatted at 2026-08-06 22:41:20 on dist-test-slave-3kk6"
server_key: "0dd97605f2d1524a1bc261be9bd11b51"
server_key_iv: "b369b6822e948180ad1636f610dca886"
server_key_version: "encryptionkey@0"
I20260806 22:41:20.861828 9409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:20.894428 9409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:20.896620 9409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:20.896833 9409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:20.897113 9409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:20.897478 9409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:20.897537 9409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:20.897583 9409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:20.897611 9409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:20.904129 9409 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:32949
I20260806 22:41:20.904177 9613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:32949 every 8 connection(s)
I20260806 22:41:20.904548 9409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:41:20.910642 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 9409
I20260806 22:41:20.910801 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:41:20.911033 17325 external_mini_cluster.cc:1428] Setting key 27f35c2fd8fb786031e84b94b1fb317b
Aug 06 22:41:20 dist-test-slave-3kk6 krb5kdc[9259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056080, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:20.914911 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:20.905476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:44207 (local address 127.16.235.126:38157)
0806 22:41:20.905548 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:20.905550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:20.905746 (+ 196us) server_negotiation.cc:407] Connection header received
0806 22:41:20.906246 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:20.906247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:20.906259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:20.906325 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:20.907428 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:20.907958 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:20.908966 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:20.909179 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:20.909234 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:20.911820 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:20.911821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:20.911822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:20.911830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:20.913362 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:20.913826 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:20.913827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:20.913828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:20.913892 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:20.914174 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:20.914175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:20.914176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:20.914302 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:20.914383 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:20.914709 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 22:41:20.914816 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:41:20.915113 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:20.905432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:38157 (local address 127.16.235.65:44207)
0806 22:41:20.905601 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:20.905646 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:20.906153 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:20.906383 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:20.906386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:20.907093 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:20.907311 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:20.907317 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:20.908061 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:20.908062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:20.908877 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:20.908879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:20.909262 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:20.909263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:20.909369 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:20.909894 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:20.909903 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:20.911735 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:20.913471 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:20.913481 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:20.913483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:20.913727 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:20.913960 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:20.913961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:20.913961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:20.914100 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:20.914375 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:20.914383 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:20.914539 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 22:41:20.914648 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:41:20.914855 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260806 22:41:20.915609 9614 heartbeater.cc:347] Connected to a master server at 127.16.235.126:38157
I20260806 22:41:20.915721 9614 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:20.915915 9614 heartbeater.cc:511] Master 127.16.235.126:38157 requested a full tablet report, sending...
I20260806 22:41:20.916553 9323 ts_manager.cc:194] Registered new tserver with Master: 648e4ac8d33548eb9048e4772a8a47b5 (127.16.235.65:32949)
I20260806 22:41:20.917202 9323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:44207
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:41:20.941638 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38157
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:21.029560 9620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:21.029819 9620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:21.029865 9620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:21.031129 9620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:21.031199 9620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:21.031248 9620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:41:21.032840 9620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:21.033143 9620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:21.033392 9620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:21.038722 9627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:21.038713 9630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:21.038729 9626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:21.038923 9620 server_base.cc:1034] running on GCE node
I20260806 22:41:21.039134 9620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:21.039389 9620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:21.040572 9620 hybrid_clock.cc:648] HybridClock initialized: now 1786056081040554 us; error 33 us; skew 500 ppm
Aug 06 22:41:21 dist-test-slave-3kk6 krb5kdc[9259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056081, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:21.043373 9620 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:41:21.043884 9620 webserver.cc:460] Webserver started at http://127.16.235.66:44339/ using document root <none> and password file <none>
I20260806 22:41:21.044116 9620 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:21.044159 9620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:21.044257 9620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:21.045313 9620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "2a51da8742ee45579bb8dd617ae3fe45"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "6e2e9bb60b326f5353836f8a41d057ed"
server_key_iv: "bdf1166c07525eb92eb26445d1621706"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.045647 9620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "2a51da8742ee45579bb8dd617ae3fe45"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "6e2e9bb60b326f5353836f8a41d057ed"
server_key_iv: "bdf1166c07525eb92eb26445d1621706"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.047101 9620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:41:21.047945 9638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.048121 9620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:21.048215 9620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "2a51da8742ee45579bb8dd617ae3fe45"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "6e2e9bb60b326f5353836f8a41d057ed"
server_key_iv: "bdf1166c07525eb92eb26445d1621706"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.048280 9620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:21.063468 9620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:21.065766 9620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:21.065953 9620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:21.066179 9620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:21.066531 9620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:21.066567 9620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.066612 9620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:21.066640 9620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.073120 9620 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:45631
I20260806 22:41:21.073163 9813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:45631 every 8 connection(s)
I20260806 22:41:21.073545 9620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:41:21.078249 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 9620
I20260806 22:41:21.078387 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:41:21.078620 17325 external_mini_cluster.cc:1428] Setting key 4404b19c2118457979a945a06bfa7dc7
Aug 06 22:41:21 dist-test-slave-3kk6 krb5kdc[9259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056081, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:21.084560 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.074441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:48563 (local address 127.16.235.126:38157)
0806 22:41:21.074535 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.074537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.074814 (+ 277us) server_negotiation.cc:407] Connection header received
0806 22:41:21.075397 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.075399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.075413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.075489 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:21.076569 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.077081 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.078142 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.078399 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.078489 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.081430 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:21.081431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:21.081432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:21.081440 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:21.083106 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:21.083524 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:21.083525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:21.083526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:21.083594 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:21.083838 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:21.083839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:21.083840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:21.083996 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:21.084078 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.084338 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.084443 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:41:21.084744 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.074366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:38157 (local address 127.16.235.66:48563)
0806 22:41:21.074612 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:21.074688 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:21.075272 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:21.075573 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:21.075577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:21.076222 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:21.076434 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.076439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.077183 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.077185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.078019 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.078021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.078527 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.078529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.078734 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.079410 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:21.079422 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:21.081316 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:21.083193 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:21.083200 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:21.083202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:21.083437 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:21.083667 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:21.083667 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:21.083668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:21.083771 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:21.084067 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:21.084072 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:21.084194 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 22:41:21.084281 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:41:21.084420 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":106,"threads_started":1}
I20260806 22:41:21.085258 9815 heartbeater.cc:347] Connected to a master server at 127.16.235.126:38157
I20260806 22:41:21.085362 9815 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:21.085583 9815 heartbeater.cc:511] Master 127.16.235.126:38157 requested a full tablet report, sending...
I20260806 22:41:21.086055 9323 ts_manager.cc:194] Registered new tserver with Master: 2a51da8742ee45579bb8dd617ae3fe45 (127.16.235.66:45631)
I20260806 22:41:21.086411 9323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:48563
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:41:21.110913 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38157
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:21.202073 9822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:21.202337 9822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:21.202382 9822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:21.203702 9822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:21.203784 9822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:21.203836 9822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:41:21.205549 9822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:38157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:21.205874 9822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:21.206136 9822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:21.211537 9836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:21.211633 9833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:21.211539 9831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:21.211796 9822 server_base.cc:1034] running on GCE node
I20260806 22:41:21.212100 9822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:21.212366 9822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:21.213527 9822 hybrid_clock.cc:648] HybridClock initialized: now 1786056081213512 us; error 31 us; skew 500 ppm
Aug 06 22:41:21 dist-test-slave-3kk6 krb5kdc[9259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056081, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:21.216563 9822 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:41:21.217077 9822 webserver.cc:460] Webserver started at http://127.16.235.67:34457/ using document root <none> and password file <none>
I20260806 22:41:21.217337 9822 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:21.217419 9822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:21.217581 9822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:21.218830 9822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "4241757efc9b48e5a6f25ec29a0a864d"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "44c96ee96957e5aff1fc144111e22fc3"
server_key_iv: "ad93afd79fba56933ade50b5b8efd7b6"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.219223 9822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "4241757efc9b48e5a6f25ec29a0a864d"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "44c96ee96957e5aff1fc144111e22fc3"
server_key_iv: "ad93afd79fba56933ade50b5b8efd7b6"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.220723 9822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:21.221567 9847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.221799 9822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:21.221915 9822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "4241757efc9b48e5a6f25ec29a0a864d"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "44c96ee96957e5aff1fc144111e22fc3"
server_key_iv: "ad93afd79fba56933ade50b5b8efd7b6"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.221999 9822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:21.236864 9822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:21.239579 9822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:21.239771 9822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:21.240047 9822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:21.240425 9822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:21.240492 9822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.240540 9822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:21.240564 9822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.247155 9822 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:43245
I20260806 22:41:21.247207 10006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:43245 every 8 connection(s)
I20260806 22:41:21.247627 9822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:41:21 dist-test-slave-3kk6 krb5kdc[9259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056081, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:21.257426 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.248460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:52771 (local address 127.16.235.126:38157)
0806 22:41:21.248508 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.248510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.248758 (+ 248us) server_negotiation.cc:407] Connection header received
0806 22:41:21.249323 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.249325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.249340 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.249404 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:21.250468 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.251058 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.252130 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.252402 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.252482 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.254778 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:21.254780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:21.254780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:21.254787 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:21.256248 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:21.256622 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:21.256623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:21.256623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:21.256676 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:21.256890 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:21.256891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:21.256891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:21.257005 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:21.257058 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.257289 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.257353 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:41:21.257653 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 9822
I20260806 22:41:21.257699 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.248473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:38157 (local address 127.16.235.67:52771)
0806 22:41:21.248619 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:21.248658 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:21.249202 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:21.249488 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:21.249491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:21.250140 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:21.250352 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.250358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.251162 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.251163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.252012 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.252015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.252489 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.252491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.252627 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.253142 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:21.253151 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:21.254632 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:21.256327 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:21.256333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:21.256335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:21.256554 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:21.256736 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:21.256737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:21.256737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:21.256836 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:21.257061 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:21.257065 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:21.257177 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 22:41:21.257258 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:41:21.257392 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260806 22:41:21.257766 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:41:21.257970 17325 external_mini_cluster.cc:1428] Setting key 6ee344c3437dcf85dbd63e6b3bc805e9
I20260806 22:41:21.258360 10007 heartbeater.cc:347] Connected to a master server at 127.16.235.126:38157
I20260806 22:41:21.258507 10007 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:21.258749 10007 heartbeater.cc:511] Master 127.16.235.126:38157 requested a full tablet report, sending...
I20260806 22:41:21.259158 9323 ts_manager.cc:194] Registered new tserver with Master: 4241757efc9b48e5a6f25ec29a0a864d (127.16.235.67:43245)
I20260806 22:41:21.259552 9323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:52771
I20260806 22:41:21.270076 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:41:21.277514 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.271575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.16.235.126:38157)
0806 22:41:21.271644 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.271647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.271790 (+ 143us) server_negotiation.cc:407] Connection header received
0806 22:41:21.271851 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.271851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.271861 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.271923 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:21.272464 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.272951 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.273640 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.273822 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.273872 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.275103 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:21.275104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:21.275105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:21.275112 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:21.276549 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:21.276855 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:21.276856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:21.276856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:21.276918 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:21.277083 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:21.277084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:21.277084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:21.277229 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:21.277307 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.277386 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.277448 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:41:21.279330 9323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:41:21.280929 9323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:41:21.288450 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.283967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35086 (local address 127.16.235.66:45631)
0806 22:41:21.284185 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.284188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.284200 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:41:21.284247 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.284249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.284279 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.284566 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:21.284644 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.285793 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.287454 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.287965 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.288058 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.288121 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.288289 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.288338 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":113,"threads_started":1}
I20260806 22:41:21.288436 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.283448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:45631 (local address 127.0.0.1:35086)
0806 22:41:21.283868 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:21.283919 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:21.283996 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:21.284396 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:21.284400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:21.284402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:21.284484 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.284487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.285929 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.285930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.287346 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.287349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.288072 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.288073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.288170 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.288184 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:41:21.288270 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:41:21.288356 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":34,"thread_start_us":78,"threads_started":1}
I20260806 22:41:21.288897 10023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.283657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52066 (local address 127.16.235.65:32949)
0806 22:41:21.284208 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.284213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.284390 (+ 177us) server_negotiation.cc:407] Connection header received
0806 22:41:21.284434 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.284436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.284451 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.284534 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:21.284684 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.285887 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.287219 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.287985 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.288059 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.288117 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.288748 (+ 631us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.288812 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":94,"threads_started":1}
I20260806 22:41:21.289050 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.283475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:32949 (local address 127.0.0.1:52066)
0806 22:41:21.284260 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:21.284270 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:21.284328 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:21.284547 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:21.284548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:21.284550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:21.284605 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.284607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.285982 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.285983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.287108 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.287112 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.288152 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.288153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.288218 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.288221 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:21.288266 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:41:21.288343 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":62,"thread_start_us":107,"threads_started":1}
I20260806 22:41:21.289660 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.283689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:43245 (local address 127.0.0.1:50484)
0806 22:41:21.285495 (+ 1806us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:21.285507 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:21.285572 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:21.285898 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:21.285900 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:21.285902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:21.285954 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.285955 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.287293 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.287294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.288335 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.288338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.289425 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.289426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.289489 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.289494 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:41:21.289548 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:41:21.289595 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1789,"thread_start_us":88,"threads_started":1}
I20260806 22:41:21.289772 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.283890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50484 (local address 127.16.235.67:43245)
0806 22:41:21.284101 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.284107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.285644 (+ 1537us) server_negotiation.cc:407] Connection header received
0806 22:41:21.285700 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.285703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.285721 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.285800 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:21.286058 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.287198 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.288432 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.289339 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.289423 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.289480 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.289593 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.289637 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260806 22:41:21.289675 9708 tablet_service.cc:1467] Processing CreateTablet for tablet 442aa78ed0344136ba217bbe68644153 (DEFAULT_TABLE table=test-table [id=3c2d3e8a62d5441e8aeb8237a05384ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:21.289894 9505 tablet_service.cc:1467] Processing CreateTablet for tablet 442aa78ed0344136ba217bbe68644153 (DEFAULT_TABLE table=test-table [id=3c2d3e8a62d5441e8aeb8237a05384ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:21.290110 9708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 442aa78ed0344136ba217bbe68644153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:21.290230 9505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 442aa78ed0344136ba217bbe68644153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:21.291392 9914 tablet_service.cc:1467] Processing CreateTablet for tablet 442aa78ed0344136ba217bbe68644153 (DEFAULT_TABLE table=test-table [id=3c2d3e8a62d5441e8aeb8237a05384ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:21.291738 9914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 442aa78ed0344136ba217bbe68644153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:21.293854 10033 tablet_bootstrap.cc:492] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5: Bootstrap starting.
I20260806 22:41:21.294281 10034 tablet_bootstrap.cc:492] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: Bootstrap starting.
I20260806 22:41:21.294723 10033 tablet_bootstrap.cc:654] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:21.294829 10030 tablet_bootstrap.cc:492] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: Bootstrap starting.
I20260806 22:41:21.295035 10034 tablet_bootstrap.cc:654] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:21.295459 10033 log.cc:824] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:21.295622 10030 tablet_bootstrap.cc:654] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:21.295732 10034 log.cc:824] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:21.296211 10030 log.cc:824] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:21.296262 10033 tablet_bootstrap.cc:492] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5: No bootstrap required, opened a new log
I20260806 22:41:21.296352 10033 ts_tablet_manager.cc:1397] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 22:41:21.296579 10034 tablet_bootstrap.cc:492] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: No bootstrap required, opened a new log
I20260806 22:41:21.296662 10034 ts_tablet_manager.cc:1397] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:41:21.296970 10030 tablet_bootstrap.cc:492] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: No bootstrap required, opened a new log
I20260806 22:41:21.297045 10030 ts_tablet_manager.cc:1397] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:41:21.297935 10033 raft_consensus.cc:348] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } }
I20260806 22:41:21.298117 10033 raft_consensus.cc:374] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:21.298166 10033 raft_consensus.cc:729] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648e4ac8d33548eb9048e4772a8a47b5, State: Initialized, Role: FOLLOWER
I20260806 22:41:21.298280 10033 consensus_queue.cc:260] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } }
I20260806 22:41:21.298565 10033 ts_tablet_manager.cc:1428] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:41:21.298570 9614 heartbeater.cc:503] Master 127.16.235.126:38157 was elected leader, sending a full tablet report...
I20260806 22:41:21.298748 10034 raft_consensus.cc:348] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } }
I20260806 22:41:21.298946 10034 raft_consensus.cc:374] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:21.298988 10034 raft_consensus.cc:729] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4241757efc9b48e5a6f25ec29a0a864d, State: Initialized, Role: FOLLOWER
I20260806 22:41:21.299099 10034 consensus_queue.cc:260] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } }
I20260806 22:41:21.299119 10030 raft_consensus.cc:348] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } }
I20260806 22:41:21.299283 10030 raft_consensus.cc:374] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:21.299321 10030 raft_consensus.cc:729] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a51da8742ee45579bb8dd617ae3fe45, State: Initialized, Role: FOLLOWER
I20260806 22:41:21.299386 10034 ts_tablet_manager.cc:1428] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:41:21.299446 10030 consensus_queue.cc:260] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } }
I20260806 22:41:21.299496 10007 heartbeater.cc:503] Master 127.16.235.126:38157 was elected leader, sending a full tablet report...
I20260806 22:41:21.299749 9815 heartbeater.cc:503] Master 127.16.235.126:38157 was elected leader, sending a full tablet report...
I20260806 22:41:21.300072 10030 ts_tablet_manager.cc:1428] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 22:41:21.324349 9817 tablet.cc:2369] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:41:21.405411 9615 tablet.cc:2369] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:41:21.498387 10008 tablet.cc:2369] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:21.526348 10042 raft_consensus.cc:482] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:21.526506 10042 raft_consensus.cc:504] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } }
I20260806 22:41:21.527041 10042 leader_election.cc:288] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a51da8742ee45579bb8dd617ae3fe45 (127.16.235.66:45631), 648e4ac8d33548eb9048e4772a8a47b5 (127.16.235.65:32949)
I20260806 22:41:21.530184 10023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.527362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:46807 (local address 127.16.235.65:32949)
0806 22:41:21.527406 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.527408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.527420 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:41:21.527527 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.527528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.527542 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.527603 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:21.527813 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.528371 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.529393 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.529761 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.529825 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.529834 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.530065 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.530110 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:41:21.530203 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.527359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:57991 (local address 127.16.235.66:45631)
0806 22:41:21.527423 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.527425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.527604 (+ 179us) server_negotiation.cc:407] Connection header received
0806 22:41:21.527637 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.527638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.527651 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.527727 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:21.527960 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.528435 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.529431 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.529832 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.529910 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.529919 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.530096 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.530142 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:41:21.530185 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.527296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:32949 (local address 127.16.235.67:46807)
0806 22:41:21.527361 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:21.527374 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:21.527430 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:21.527631 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:21.527632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:21.527635 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:21.527721 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.527724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.528466 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.528467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.529281 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.529284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.529854 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.529855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.529963 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.529967 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:21.530022 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:41:21.530089 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":21}
I20260806 22:41:21.530200 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.527304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:45631 (local address 127.16.235.67:57991)
0806 22:41:21.527498 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:21.527510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:21.527574 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:21.527740 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:21.527741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:21.527743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:21.527871 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.527873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.528526 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.528527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.529293 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.529295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.529924 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.529925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.529982 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.529985 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:21.530069 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:41:21.530110 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":17,"thread_start_us":119,"threads_started":1}
I20260806 22:41:21.530630 9538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442aa78ed0344136ba217bbe68644153" candidate_uuid: "4241757efc9b48e5a6f25ec29a0a864d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648e4ac8d33548eb9048e4772a8a47b5" is_pre_election: true
I20260806 22:41:21.530691 9738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442aa78ed0344136ba217bbe68644153" candidate_uuid: "4241757efc9b48e5a6f25ec29a0a864d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a51da8742ee45579bb8dd617ae3fe45" is_pre_election: true
I20260806 22:41:21.530875 9538 raft_consensus.cc:2454] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4241757efc9b48e5a6f25ec29a0a864d in term 0.
I20260806 22:41:21.530885 9738 raft_consensus.cc:2454] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4241757efc9b48e5a6f25ec29a0a864d in term 0.
I20260806 22:41:21.531174 9851 leader_election.cc:302] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a51da8742ee45579bb8dd617ae3fe45, 4241757efc9b48e5a6f25ec29a0a864d; no voters:
I20260806 22:41:21.531334 10042 raft_consensus.cc:2781] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:41:21.531399 10042 raft_consensus.cc:482] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:41:21.531424 10042 raft_consensus.cc:3037] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:21.532236 10042 raft_consensus.cc:504] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } }
I20260806 22:41:21.532403 10042 leader_election.cc:288] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [CANDIDATE]: Term 1 election: Requested vote from peers 2a51da8742ee45579bb8dd617ae3fe45 (127.16.235.66:45631), 648e4ac8d33548eb9048e4772a8a47b5 (127.16.235.65:32949)
I20260806 22:41:21.532577 9738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442aa78ed0344136ba217bbe68644153" candidate_uuid: "4241757efc9b48e5a6f25ec29a0a864d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a51da8742ee45579bb8dd617ae3fe45"
I20260806 22:41:21.532616 9538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442aa78ed0344136ba217bbe68644153" candidate_uuid: "4241757efc9b48e5a6f25ec29a0a864d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648e4ac8d33548eb9048e4772a8a47b5"
I20260806 22:41:21.532668 9738 raft_consensus.cc:3037] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:21.532716 9538 raft_consensus.cc:3037] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:21.533476 9738 raft_consensus.cc:2454] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4241757efc9b48e5a6f25ec29a0a864d in term 1.
I20260806 22:41:21.533481 9538 raft_consensus.cc:2454] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4241757efc9b48e5a6f25ec29a0a864d in term 1.
I20260806 22:41:21.533658 9851 leader_election.cc:302] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a51da8742ee45579bb8dd617ae3fe45, 4241757efc9b48e5a6f25ec29a0a864d; no voters:
I20260806 22:41:21.533792 10042 raft_consensus.cc:2781] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:21.534024 10042 raft_consensus.cc:686] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 1 LEADER]: Becoming Leader. State: Replica: 4241757efc9b48e5a6f25ec29a0a864d, State: Running, Role: LEADER
I20260806 22:41:21.534142 10042 consensus_queue.cc:237] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } }
I20260806 22:41:21.535070 9323 catalog_manager.cc:5495] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4241757efc9b48e5a6f25ec29a0a864d (127.16.235.67). New cstate: current_term: 1 leader_uuid: "4241757efc9b48e5a6f25ec29a0a864d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4241757efc9b48e5a6f25ec29a0a864d" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43245 } health_report { overall_health: HEALTHY } } }
I20260806 22:41:21.572171 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.569524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50486 (local address 127.16.235.67:43245)
0806 22:41:21.569595 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.569597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.569611 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:41:21.569652 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.569653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.569672 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.569735 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:41:21.569882 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.570366 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.571245 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.571444 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.571502 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.571719 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:41:21.571856 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:41:21.571935 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.572032 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.572052 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:41:21.574328 9738 raft_consensus.cc:1261] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [term 1 FOLLOWER]: Refusing update from remote peer 4241757efc9b48e5a6f25ec29a0a864d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:21.574361 9538 raft_consensus.cc:1261] T 442aa78ed0344136ba217bbe68644153 P 648e4ac8d33548eb9048e4772a8a47b5 [term 1 FOLLOWER]: Refusing update from remote peer 4241757efc9b48e5a6f25ec29a0a864d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:21.574838 10045 consensus_queue.cc:1035] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a51da8742ee45579bb8dd617ae3fe45" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 45631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:21.575029 10042 consensus_queue.cc:1035] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [LEADER]: Connected to new peer: Peer: permanent_uuid: "648e4ac8d33548eb9048e4772a8a47b5" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 32949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:21.578172 10052 mvcc.cc:205] Tried to move back new op lower bound from 7315685710125289472 to 7315685709964529664. Current Snapshot: MvccSnapshot[applied={T|T < 7315685710125289472}]
I20260806 22:41:21.578353 10048 mvcc.cc:205] Tried to move back new op lower bound from 7315685710125289472 to 7315685709964529664. Current Snapshot: MvccSnapshot[applied={T|T < 7315685710125289472 or (T in {7315685710125289472})}]
I20260806 22:41:21.578408 10051 mvcc.cc:205] Tried to move back new op lower bound from 7315685710125289472 to 7315685709964529664. Current Snapshot: MvccSnapshot[applied={T|T < 7315685710125289472}]
I20260806 22:41:21.580725 9322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:21.580845 9322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:21.581753 9322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da9bded7485c4bd28a6f48403cab5fa9: Sending DeleteTablet for 3 replicas of tablet 442aa78ed0344136ba217bbe68644153
I20260806 22:41:21.582154 9505 tablet_service.cc:1514] Processing DeleteTablet for tablet 442aa78ed0344136ba217bbe68644153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:21 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:52066
I20260806 22:41:21.582152 9708 tablet_service.cc:1514] Processing DeleteTablet for tablet 442aa78ed0344136ba217bbe68644153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:21 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:35086
I20260806 22:41:21.582394 9914 tablet_service.cc:1514] Processing DeleteTablet for tablet 442aa78ed0344136ba217bbe68644153 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:21 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:50484
I20260806 22:41:21.582617 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 9409
I20260806 22:41:21.590556 10073 tablet_replica.cc:330] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: stopping tablet replica
I20260806 22:41:21.590745 10073 raft_consensus.cc:2229] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:41:21.590878 10073 raft_consensus.cc:2258] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:21.591233 10071 tablet_replica.cc:330] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: stopping tablet replica
I20260806 22:41:21.591374 10071 raft_consensus.cc:2229] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:41:21.591533 10071 raft_consensus.cc:2258] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:21.591619 10073 ts_tablet_manager.cc:1905] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:21.593842 10071 ts_tablet_manager.cc:1905] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:21.594023 10073 ts_tablet_manager.cc:1918] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:21.594126 10073 log.cc:1196] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/442aa78ed0344136ba217bbe68644153
I20260806 22:41:21.594545 10073 ts_tablet_manager.cc:1939] T 442aa78ed0344136ba217bbe68644153 P 4241757efc9b48e5a6f25ec29a0a864d: Deleting consensus metadata
I20260806 22:41:21.595243 9307 catalog_manager.cc:4841] TS 4241757efc9b48e5a6f25ec29a0a864d (127.16.235.67:43245): tablet 442aa78ed0344136ba217bbe68644153 (table test-table [id=3c2d3e8a62d5441e8aeb8237a05384ca]) successfully deleted
I20260806 22:41:21.595829 10071 ts_tablet_manager.cc:1918] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:21.595939 10071 log.cc:1196] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/442aa78ed0344136ba217bbe68644153
I20260806 22:41:21.596233 10071 ts_tablet_manager.cc:1939] T 442aa78ed0344136ba217bbe68644153 P 2a51da8742ee45579bb8dd617ae3fe45: Deleting consensus metadata
I20260806 22:41:21.596755 9307 catalog_manager.cc:4841] TS 2a51da8742ee45579bb8dd617ae3fe45 (127.16.235.66:45631): tablet 442aa78ed0344136ba217bbe68644153 (table test-table [id=3c2d3e8a62d5441e8aeb8237a05384ca]) successfully deleted
W20260806 22:41:21.601480 9305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:32949 (error 108)
I20260806 22:41:21.602165 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.602022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:32949 (local address 127.0.0.1:52082)
0806 22:41:21.602083 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:21.602134 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:32949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:41:21.602322 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 9620
W20260806 22:41:21.602324 9305 catalog_manager.cc:4584] TS 648e4ac8d33548eb9048e4772a8a47b5 (127.16.235.65:32949): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 442aa78ed0344136ba217bbe68644153: Network error: Client connection negotiation failed: client connection to 127.16.235.65:32949: connect: Connection refused (error 111)
I20260806 22:41:21.618824 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 9822
I20260806 22:41:21.630605 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 9281
2026-08-06T22:41:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2476 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T22:41:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:21Z Disabled control of system clock
I20260806 22:41:21.666105 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:35935
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:35935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:41:21.751775 10078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:21.752025 10078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:21.752048 10078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:21.753228 10078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:21.753275 10078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:21.753289 10078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:21.753301 10078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:21.753314 10078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:21.754822 10078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:35935
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:35935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:21.755102 10078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:21.755422 10078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:21.760790 10085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:21.760794 10089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:21.760918 10087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:21.761137 10078 server_base.cc:1034] running on GCE node
I20260806 22:41:21.761421 10078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:21.761708 10078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:21.762900 10078 hybrid_clock.cc:648] HybridClock initialized: now 1786056081762883 us; error 35 us; skew 500 ppm
I20260806 22:41:21.764308 10078 webserver.cc:460] Webserver started at http://127.16.235.126:40941/ using document root <none> and password file <none>
I20260806 22:41:21.764595 10078 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:21.764679 10078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:21.764829 10078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:21.766064 10078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "f9448238377d49d3a568278254fc1f50"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "2de84c66f001cf165a2f8151fcdee0ab"
server_key_iv: "8a602863807d60122481dd1f3eb509e7"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.766516 10078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "f9448238377d49d3a568278254fc1f50"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "2de84c66f001cf165a2f8151fcdee0ab"
server_key_iv: "8a602863807d60122481dd1f3eb509e7"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.768014 10078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:21.768831 10101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.769011 10078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:41:21.769133 10078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "f9448238377d49d3a568278254fc1f50"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "2de84c66f001cf165a2f8151fcdee0ab"
server_key_iv: "8a602863807d60122481dd1f3eb509e7"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.769232 10078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:21.778755 10078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:21.779139 10078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:21.779341 10078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:21.784054 10078 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:35935
I20260806 22:41:21.784092 10172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:35935 every 8 connection(s)
I20260806 22:41:21.784523 10078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:21.785640 10174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:21.788240 10174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50: Bootstrap starting.
I20260806 22:41:21.788913 10174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:21.789248 10174 log.cc:824] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:21.790043 10174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50: No bootstrap required, opened a new log
I20260806 22:41:21.791754 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 10078
I20260806 22:41:21.791870 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:21.792096 17325 external_mini_cluster.cc:1428] Setting key 07c2664cda2be53c7005ab7bd6f4ca81
I20260806 22:41:21.792001 10174 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9448238377d49d3a568278254fc1f50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35935 } }
I20260806 22:41:21.792191 10174 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:21.792243 10174 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9448238377d49d3a568278254fc1f50, State: Initialized, Role: FOLLOWER
I20260806 22:41:21.792393 10174 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9448238377d49d3a568278254fc1f50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35935 } }
I20260806 22:41:21.792505 10174 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:21.792543 10174 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:21.792593 10174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:21.793738 10174 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9448238377d49d3a568278254fc1f50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35935 } }
I20260806 22:41:21.793865 10174 leader_election.cc:302] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9448238377d49d3a568278254fc1f50; no voters:
I20260806 22:41:21.794075 10174 leader_election.cc:288] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:21.794190 10183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:21.794342 10174 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:21.794379 10183 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [term 1 LEADER]: Becoming Leader. State: Replica: f9448238377d49d3a568278254fc1f50, State: Running, Role: LEADER
I20260806 22:41:21.794603 10183 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9448238377d49d3a568278254fc1f50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35935 } }
I20260806 22:41:21.795091 10185 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9448238377d49d3a568278254fc1f50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9448238377d49d3a568278254fc1f50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35935 } } }
I20260806 22:41:21.795177 10185 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:21.795316 10185 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9448238377d49d3a568278254fc1f50. Latest consensus state: current_term: 1 leader_uuid: "f9448238377d49d3a568278254fc1f50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9448238377d49d3a568278254fc1f50" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 35935 } } }
I20260806 22:41:21.795392 10185 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:21.795863 10195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:21.796530 10195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:21.798400 10195 catalog_manager.cc:1347] Generated new cluster ID: 588b41cee2d34a63a159484eda9e9620
I20260806 22:41:21.798480 10195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:21.799006 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.792688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59142 (local address 127.16.235.126:35935)
0806 22:41:21.792960 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.792969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.793005 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:41:21.793139 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.793146 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.793216 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.793366 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:21.793588 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.795624 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.796724 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.797146 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.797288 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.798108 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:21.798112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:21.798112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:21.798336 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:21.798382 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:21.798398 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:21.798481 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.798637 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.798770 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":140,"threads_started":1}
I20260806 22:41:21.812536 10195 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:21.813336 10195 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:21.818353 10195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50: Generated new TSK 0
I20260806 22:41:21.818599 10195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:21.824378 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35935
--builtin_ntp_servers=127.16.235.84:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:41:21.919674 10213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:21.919912 10213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:21.919936 10213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:21.921212 10213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:21.921260 10213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:21.921288 10213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:41:21.922843 10213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:21.923135 10213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:21.923508 10213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:21.929853 10222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:21.929849 10224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:21.930150 10213 server_base.cc:1034] running on GCE node
W20260806 22:41:21.929853 10221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:21.930440 10213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:21.930784 10213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:21.931962 10213 hybrid_clock.cc:648] HybridClock initialized: now 1786056081931938 us; error 52 us; skew 500 ppm
I20260806 22:41:21.933346 10213 webserver.cc:460] Webserver started at http://127.16.235.65:42883/ using document root <none> and password file <none>
I20260806 22:41:21.933640 10213 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:21.933717 10213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:21.933877 10213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:21.935137 10213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "f3ac07af5e5a409d93de7a09ab8476bd"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "34578b9c92904fe21b7ee2aca4fd0ae1"
server_key_iv: "d9dabef48a2df7a25b05322c155f53a8"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.935534 10213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "f3ac07af5e5a409d93de7a09ab8476bd"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "34578b9c92904fe21b7ee2aca4fd0ae1"
server_key_iv: "d9dabef48a2df7a25b05322c155f53a8"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.937016 10213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:41:21.937848 10233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.938041 10213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:21.938139 10213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "f3ac07af5e5a409d93de7a09ab8476bd"
format_stamp: "Formatted at 2026-08-06 22:41:21 on dist-test-slave-3kk6"
server_key: "34578b9c92904fe21b7ee2aca4fd0ae1"
server_key_iv: "d9dabef48a2df7a25b05322c155f53a8"
server_key_version: "encryptionkey@0"
I20260806 22:41:21.938228 10213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:21.949630 10213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:21.950008 10213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:21.950186 10213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:21.950457 10213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:21.950960 10213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:21.951023 10213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.951058 10213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:21.951102 10213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:21.957692 10213 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:39727
I20260806 22:41:21.957731 10400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:39727 every 8 connection(s)
I20260806 22:41:21.958130 10213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:41:21.961628 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 10213
I20260806 22:41:21.961757 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:41:21.961979 17325 external_mini_cluster.cc:1428] Setting key 1e7da1b6b8ba65c83154c8868ed720cb
I20260806 22:41:21.963111 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35935
--builtin_ntp_servers=127.16.235.84:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:41:21.964231 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.959106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:42043 (local address 127.16.235.126:35935)
0806 22:41:21.959177 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:21.959179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:21.959461 (+ 282us) server_negotiation.cc:407] Connection header received
0806 22:41:21.960036 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:21.960037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:21.960053 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:21.960120 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:21.960859 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.961381 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.962572 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.962880 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.962948 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.963958 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:21.963959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:21.963960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:21.963964 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:21.963979 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:21.963983 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:21.964029 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:21.964133 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:41:21.964165 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:41:21.964525 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:21.959112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:35935 (local address 127.16.235.65:42043)
0806 22:41:21.959330 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:21.959367 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:21.959918 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:21.960203 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:21.960206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:21.960214 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:21.960743 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.960748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.961535 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.961536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.962485 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:21.962488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:21.962986 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:21.962987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:21.963162 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:21.963791 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:41:21.963792 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:21.963833 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:41:21.963863 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:21.964048 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:21.964056 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:21.964058 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:41:21.964129 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:41:21.964252 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260806 22:41:21.965384 10401 heartbeater.cc:347] Connected to a master server at 127.16.235.126:35935
I20260806 22:41:21.965481 10401 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:21.965687 10401 heartbeater.cc:511] Master 127.16.235.126:35935 requested a full tablet report, sending...
I20260806 22:41:21.966324 10121 ts_manager.cc:194] Registered new tserver with Master: f3ac07af5e5a409d93de7a09ab8476bd (127.16.235.65:39727)
I20260806 22:41:21.967059 10121 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.65:42043
W20260806 22:41:22.051568 10405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:22.051875 10405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:22.051915 10405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:22.053622 10405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:22.053699 10405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:22.053742 10405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:41:22.055203 10405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:22.055478 10405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:22.055800 10405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:22.061079 10413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:22.061103 10412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:22.061344 10405 server_base.cc:1034] running on GCE node
W20260806 22:41:22.061097 10416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:22.061725 10405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:22.062011 10405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:22.063171 10405 hybrid_clock.cc:648] HybridClock initialized: now 1786056082063168 us; error 38 us; skew 500 ppm
I20260806 22:41:22.064494 10405 webserver.cc:460] Webserver started at http://127.16.235.66:35181/ using document root <none> and password file <none>
I20260806 22:41:22.064754 10405 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:22.064851 10405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:22.065002 10405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:22.066156 10405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "484db560bab64a859b234b7c43f2e13f"
format_stamp: "Formatted at 2026-08-06 22:41:22 on dist-test-slave-3kk6"
server_key: "a116505ea37c41d1f3e0c82ea502991d"
server_key_iv: "8ca2bd8c700572975791af2648e96142"
server_key_version: "encryptionkey@0"
I20260806 22:41:22.066551 10405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "484db560bab64a859b234b7c43f2e13f"
format_stamp: "Formatted at 2026-08-06 22:41:22 on dist-test-slave-3kk6"
server_key: "a116505ea37c41d1f3e0c82ea502991d"
server_key_iv: "8ca2bd8c700572975791af2648e96142"
server_key_version: "encryptionkey@0"
I20260806 22:41:22.068034 10405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:22.068804 10428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:22.068986 10405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:41:22.069097 10405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "484db560bab64a859b234b7c43f2e13f"
format_stamp: "Formatted at 2026-08-06 22:41:22 on dist-test-slave-3kk6"
server_key: "a116505ea37c41d1f3e0c82ea502991d"
server_key_iv: "8ca2bd8c700572975791af2648e96142"
server_key_version: "encryptionkey@0"
I20260806 22:41:22.069178 10405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:22.091708 10405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:22.092097 10405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:22.092275 10405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:22.092551 10405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:22.092943 10405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:22.093004 10405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:22.093047 10405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:22.093096 10405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:22.101112 10405 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:39681
I20260806 22:41:22.101166 10587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:39681 every 8 connection(s)
I20260806 22:41:22.101725 10405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:41:22.108270 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.103463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:58783 (local address 127.16.235.126:35935)
0806 22:41:22.103550 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:22.103552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:22.103875 (+ 323us) server_negotiation.cc:407] Connection header received
0806 22:41:22.104374 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:22.104375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:22.104392 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:22.104468 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:22.105204 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.105708 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.106832 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.107034 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.107099 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.108036 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:22.108037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:22.108037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:22.108042 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:22.108057 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:22.108062 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:22.108112 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:22.108177 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:41:22.108204 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:41:22.108517 10592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.103461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:35935 (local address 127.16.235.66:58783)
0806 22:41:22.103712 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:22.103752 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:22.104270 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:22.104523 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:22.104526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:22.104535 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:22.105091 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.105096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.105809 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.105810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.106637 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.106640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.107123 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.107160 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.107338 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.107891 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:41:22.107892 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:22.107924 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:41:22.107949 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:22.108115 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:22.108117 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:22.108120 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:22.108169 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:41:22.108255 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
I20260806 22:41:22.109055 10588 heartbeater.cc:347] Connected to a master server at 127.16.235.126:35935
I20260806 22:41:22.109184 10588 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:22.109390 10588 heartbeater.cc:511] Master 127.16.235.126:35935 requested a full tablet report, sending...
I20260806 22:41:22.109797 10121 ts_manager.cc:194] Registered new tserver with Master: 484db560bab64a859b234b7c43f2e13f (127.16.235.66:39681)
I20260806 22:41:22.110071 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 10405
I20260806 22:41:22.110190 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:41:22.110271 10121 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.66:58783
I20260806 22:41:22.110440 17325 external_mini_cluster.cc:1428] Setting key 8b3c7a7489566bfbd9cae2048f28b337
I20260806 22:41:22.111508 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35935
--builtin_ntp_servers=127.16.235.84:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:41:22.198236 10593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:22.198478 10593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:22.198503 10593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:22.200150 10593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:22.200228 10593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:22.200258 10593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:41:22.201819 10593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:35935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:22.202152 10593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:22.202528 10593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:22.208180 10598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:22.208163 10604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:22.208171 10600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:22.208508 10593 server_base.cc:1034] running on GCE node
I20260806 22:41:22.208738 10593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:22.209029 10593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:22.210189 10593 hybrid_clock.cc:648] HybridClock initialized: now 1786056082210174 us; error 42 us; skew 500 ppm
I20260806 22:41:22.211680 10593 webserver.cc:460] Webserver started at http://127.16.235.67:34491/ using document root <none> and password file <none>
I20260806 22:41:22.211969 10593 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:22.212047 10593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:22.212219 10593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:22.213409 10593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "3cd90ac26c124c44a66258644682fe74"
format_stamp: "Formatted at 2026-08-06 22:41:22 on dist-test-slave-3kk6"
server_key: "b55a8f958d47740c227d8b4728ef342b"
server_key_iv: "15e4188e5f8219d863e08a5bab4bd48b"
server_key_version: "encryptionkey@0"
I20260806 22:41:22.213814 10593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "3cd90ac26c124c44a66258644682fe74"
format_stamp: "Formatted at 2026-08-06 22:41:22 on dist-test-slave-3kk6"
server_key: "b55a8f958d47740c227d8b4728ef342b"
server_key_iv: "15e4188e5f8219d863e08a5bab4bd48b"
server_key_version: "encryptionkey@0"
I20260806 22:41:22.215606 10593 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260806 22:41:22.216557 10610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:22.216778 10593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:22.216902 10593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "3cd90ac26c124c44a66258644682fe74"
format_stamp: "Formatted at 2026-08-06 22:41:22 on dist-test-slave-3kk6"
server_key: "b55a8f958d47740c227d8b4728ef342b"
server_key_iv: "15e4188e5f8219d863e08a5bab4bd48b"
server_key_version: "encryptionkey@0"
I20260806 22:41:22.216993 10593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:22.234022 10593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:22.234416 10593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:22.234596 10593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:22.234903 10593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:22.235301 10593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:22.235368 10593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:22.235414 10593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:22.235440 10593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:22.242127 10593 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:38267
I20260806 22:41:22.242206 10775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:38267 every 8 connection(s)
I20260806 22:41:22.242607 10593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:41:22.247856 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 10593
I20260806 22:41:22.248011 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:41:22.248243 17325 external_mini_cluster.cc:1428] Setting key 9f70a5bfa76d5e260857a16d02c51e01
I20260806 22:41:22.248557 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.243500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:45161 (local address 127.16.235.126:35935)
0806 22:41:22.243566 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:22.243568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:22.243844 (+ 276us) server_negotiation.cc:407] Connection header received
0806 22:41:22.244438 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:22.244440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:22.244455 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:22.244510 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:22.245256 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.245787 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.246894 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.247105 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.247191 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.248219 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:22.248221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:22.248221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:22.248227 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:22.248244 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:22.248255 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:22.248318 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:22.248438 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:41:22.248474 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:41:22.248803 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.243461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:35935 (local address 127.16.235.67:45161)
0806 22:41:22.243712 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:22.243753 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:22.244298 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:22.244592 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:22.244596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:22.244604 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:22.245147 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.245152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.245908 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.245909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.246796 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.246799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.247203 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.247205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.247348 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.248008 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:41:22.248009 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:22.248071 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:41:22.248104 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:22.248321 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:22.248328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:22.248332 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:22.248419 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:41:22.248537 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":116,"threads_started":1}
I20260806 22:41:22.249523 10776 heartbeater.cc:347] Connected to a master server at 127.16.235.126:35935
I20260806 22:41:22.249665 10776 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:22.249897 10776 heartbeater.cc:511] Master 127.16.235.126:35935 requested a full tablet report, sending...
I20260806 22:41:22.250383 10120 ts_manager.cc:194] Registered new tserver with Master: 3cd90ac26c124c44a66258644682fe74 (127.16.235.67:38267)
I20260806 22:41:22.250849 10120 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.67:45161
I20260806 22:41:22.260668 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:41:22.265643 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.262373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.16.235.126:35935)
0806 22:41:22.262460 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:22.262463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:22.262603 (+ 140us) server_negotiation.cc:407] Connection header received
0806 22:41:22.262681 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:22.262682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:22.262699 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:22.262769 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:22.262940 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.263430 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.264244 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.264433 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.264485 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.265324 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:22.265326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:22.265326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:22.265331 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:22.265348 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:22.265353 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:22.265409 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:22.265479 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:41:22.265509 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:41:22.267978 10120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:41:22.269762 10120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:41:22.276932 10786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.272637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:39727 (local address 127.0.0.1:49750)
0806 22:41:22.273043 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:22.273054 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:22.273096 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:22.273341 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:22.273342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:22.273345 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:22.273404 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.273407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.274761 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.274763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.275789 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.275793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.276638 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.276640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.276736 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.276750 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:41:22.276823 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:41:22.276865 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":55,"thread_start_us":97,"threads_started":1}
I20260806 22:41:22.276990 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.272919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49750 (local address 127.16.235.65:39727)
0806 22:41:22.273085 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:22.273089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:22.273100 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:41:22.273216 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:22.273218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:22.273234 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:22.273326 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:22.273492 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.274674 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.275887 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.276536 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.276617 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.276711 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:22.276858 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 22:41:22.276897 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260806 22:41:22.277302 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.272387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:39681 (local address 127.0.0.1:39862)
0806 22:41:22.272794 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:22.272827 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:22.272924 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:22.273192 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:22.273195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:22.273197 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:22.273286 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.273288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.274761 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.274763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.276105 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.276108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.277057 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.277058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.277132 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.277136 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:22.277208 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:41:22.277244 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":50,"thread_start_us":71,"threads_started":1}
I20260806 22:41:22.277351 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.272482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.16.235.66:39681)
0806 22:41:22.272978 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:22.272984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:22.272997 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:41:22.273052 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:22.273056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:22.273077 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:22.273181 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:22.273383 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.274620 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.276209 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.276952 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.277029 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.277125 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:22.277223 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:41:22.277263 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":131,"threads_started":1}
I20260806 22:41:22.278115 10308 tablet_service.cc:1467] Processing CreateTablet for tablet 5f645dcbc2e0471fb2798f2d4c634c78 (DEFAULT_TABLE table=test-table [id=b3383dcc52ae46bb8216402c15e873ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:22.278199 10492 tablet_service.cc:1467] Processing CreateTablet for tablet 5f645dcbc2e0471fb2798f2d4c634c78 (DEFAULT_TABLE table=test-table [id=b3383dcc52ae46bb8216402c15e873ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:22.278573 10308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f645dcbc2e0471fb2798f2d4c634c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:22.278622 10492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f645dcbc2e0471fb2798f2d4c634c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:22.278736 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.272375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:38267 (local address 127.0.0.1:43514)
0806 22:41:22.272772 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:22.272825 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:22.272924 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:22.273456 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:22.273458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:22.273460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:22.273500 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.273501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.274594 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.274596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.276104 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.276108 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.278464 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.278465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.278531 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.278534 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:22.278589 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:41:22.278636 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":10,"thread_start_us":77,"threads_started":1}
I20260806 22:41:22.279065 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.272477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.16.235.67:38267)
0806 22:41:22.272659 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:22.272668 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:22.273210 (+ 542us) server_negotiation.cc:407] Connection header received
0806 22:41:22.273264 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:22.273267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:22.273285 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:22.273373 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:22.273592 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.274485 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.277166 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.278271 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.278339 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.278425 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:22.278911 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 22:41:22.278954 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260806 22:41:22.281865 10795 tablet_bootstrap.cc:492] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd: Bootstrap starting.
I20260806 22:41:22.282079 10794 tablet_bootstrap.cc:492] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: Bootstrap starting.
I20260806 22:41:22.282496 10674 tablet_service.cc:1467] Processing CreateTablet for tablet 5f645dcbc2e0471fb2798f2d4c634c78 (DEFAULT_TABLE table=test-table [id=b3383dcc52ae46bb8216402c15e873ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:22.282850 10794 tablet_bootstrap.cc:654] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:22.282881 10674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f645dcbc2e0471fb2798f2d4c634c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:22.283152 10794 log.cc:824] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:22.284055 10795 tablet_bootstrap.cc:654] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:22.284088 10794 tablet_bootstrap.cc:492] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: No bootstrap required, opened a new log
I20260806 22:41:22.284150 10794 ts_tablet_manager.cc:1397] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:41:22.284413 10795 log.cc:824] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:22.285329 10795 tablet_bootstrap.cc:492] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd: No bootstrap required, opened a new log
I20260806 22:41:22.285418 10795 ts_tablet_manager.cc:1397] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 22:41:22.286409 10801 tablet_bootstrap.cc:492] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: Bootstrap starting.
I20260806 22:41:22.287026 10794 raft_consensus.cc:348] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } }
I20260806 22:41:22.287184 10801 tablet_bootstrap.cc:654] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:22.287199 10794 raft_consensus.cc:374] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:22.287242 10794 raft_consensus.cc:729] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 484db560bab64a859b234b7c43f2e13f, State: Initialized, Role: FOLLOWER
I20260806 22:41:22.287397 10794 consensus_queue.cc:260] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } }
I20260806 22:41:22.287585 10801 log.cc:824] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:22.287738 10588 heartbeater.cc:503] Master 127.16.235.126:35935 was elected leader, sending a full tablet report...
I20260806 22:41:22.287725 10794 ts_tablet_manager.cc:1428] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:41:22.287979 10795 raft_consensus.cc:348] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } }
I20260806 22:41:22.288139 10795 raft_consensus.cc:374] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:22.288232 10795 raft_consensus.cc:729] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ac07af5e5a409d93de7a09ab8476bd, State: Initialized, Role: FOLLOWER
I20260806 22:41:22.288388 10795 consensus_queue.cc:260] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } }
I20260806 22:41:22.288671 10795 ts_tablet_manager.cc:1428] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:41:22.288713 10401 heartbeater.cc:503] Master 127.16.235.126:35935 was elected leader, sending a full tablet report...
I20260806 22:41:22.289242 10801 tablet_bootstrap.cc:492] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: No bootstrap required, opened a new log
I20260806 22:41:22.289348 10801 ts_tablet_manager.cc:1397] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:41:22.291436 10801 raft_consensus.cc:348] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } }
I20260806 22:41:22.291607 10801 raft_consensus.cc:374] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:22.291672 10801 raft_consensus.cc:729] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd90ac26c124c44a66258644682fe74, State: Initialized, Role: FOLLOWER
I20260806 22:41:22.291805 10801 consensus_queue.cc:260] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } }
I20260806 22:41:22.292095 10801 ts_tablet_manager.cc:1428] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:41:22.292181 10776 heartbeater.cc:503] Master 127.16.235.126:35935 was elected leader, sending a full tablet report...
W20260806 22:41:22.352519 10589 tablet.cc:2369] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:41:22.458913 10402 tablet.cc:2369] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:22.483610 10806 raft_consensus.cc:482] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:22.483834 10806 raft_consensus.cc:504] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } }
I20260806 22:41:22.484299 10806 leader_election.cc:288] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 484db560bab64a859b234b7c43f2e13f (127.16.235.66:39681), 3cd90ac26c124c44a66258644682fe74 (127.16.235.67:38267)
I20260806 22:41:22.487500 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.484712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:43261 (local address 127.16.235.67:38267)
0806 22:41:22.484762 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:22.484792 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:22.484805 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:41:22.484912 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:22.484913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:22.484924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:22.484984 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:22.485190 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.485682 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.486735 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.487124 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.487184 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.487214 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:22.487376 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:41:22.487424 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:41:22.487512 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.484554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:38267 (local address 127.16.235.65:43261)
0806 22:41:22.484714 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:22.484728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:22.484822 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:22.485008 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:22.485009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:22.485012 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:22.485112 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.485115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.485780 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.485781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.486605 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.486608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.487214 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.487216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.487297 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.487301 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:22.487347 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:41:22.487382 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260806 22:41:22.487555 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.484646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:56323 (local address 127.16.235.66:39681)
0806 22:41:22.484733 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:22.484735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:22.484840 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:41:22.484940 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:22.484942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:22.484954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:22.485030 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:22.485234 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.485689 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.486776 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.487135 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.487228 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.487237 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:22.487435 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 22:41:22.487481 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:41:22.487535 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.484554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:39681 (local address 127.16.235.65:56323)
0806 22:41:22.484723 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:22.484736 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:22.484834 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:22.485051 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:22.485051 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:22.485053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:22.485131 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.485133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.485785 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.485786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.486614 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:22.486617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.487250 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.487251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:22.487325 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.487347 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 22:41:22.487409 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:41:22.487446 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":50,"spinlock_wait_cycles":768,"thread_start_us":96,"threads_started":1}
I20260806 22:41:22.487912 10706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f645dcbc2e0471fb2798f2d4c634c78" candidate_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd90ac26c124c44a66258644682fe74" is_pre_election: true
I20260806 22:41:22.488059 10706 raft_consensus.cc:2454] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3ac07af5e5a409d93de7a09ab8476bd in term 0.
I20260806 22:41:22.488077 10520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f645dcbc2e0471fb2798f2d4c634c78" candidate_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "484db560bab64a859b234b7c43f2e13f" is_pre_election: true
I20260806 22:41:22.488195 10520 raft_consensus.cc:2454] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3ac07af5e5a409d93de7a09ab8476bd in term 0.
I20260806 22:41:22.488319 10234 leader_election.cc:302] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cd90ac26c124c44a66258644682fe74, f3ac07af5e5a409d93de7a09ab8476bd; no voters:
I20260806 22:41:22.488447 10806 raft_consensus.cc:2781] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:41:22.488548 10806 raft_consensus.cc:482] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:41:22.488588 10806 raft_consensus.cc:3037] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:22.489413 10806 raft_consensus.cc:504] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } }
I20260806 22:41:22.489588 10806 leader_election.cc:288] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [CANDIDATE]: Term 1 election: Requested vote from peers 484db560bab64a859b234b7c43f2e13f (127.16.235.66:39681), 3cd90ac26c124c44a66258644682fe74 (127.16.235.67:38267)
I20260806 22:41:22.489774 10706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f645dcbc2e0471fb2798f2d4c634c78" candidate_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd90ac26c124c44a66258644682fe74"
I20260806 22:41:22.489826 10520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f645dcbc2e0471fb2798f2d4c634c78" candidate_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "484db560bab64a859b234b7c43f2e13f"
I20260806 22:41:22.489884 10706 raft_consensus.cc:3037] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:22.489899 10520 raft_consensus.cc:3037] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:22.490773 10520 raft_consensus.cc:2454] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3ac07af5e5a409d93de7a09ab8476bd in term 1.
I20260806 22:41:22.490860 10706 raft_consensus.cc:2454] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3ac07af5e5a409d93de7a09ab8476bd in term 1.
I20260806 22:41:22.490968 10236 leader_election.cc:302] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 484db560bab64a859b234b7c43f2e13f, f3ac07af5e5a409d93de7a09ab8476bd; no voters:
I20260806 22:41:22.491065 10806 raft_consensus.cc:2781] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:22.491269 10806 raft_consensus.cc:686] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [term 1 LEADER]: Becoming Leader. State: Replica: f3ac07af5e5a409d93de7a09ab8476bd, State: Running, Role: LEADER
I20260806 22:41:22.491384 10806 consensus_queue.cc:237] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } }
I20260806 22:41:22.492358 10121 catalog_manager.cc:5495] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd reported cstate change: term changed from 0 to 1, leader changed from <none> to f3ac07af5e5a409d93de7a09ab8476bd (127.16.235.65). New cstate: current_term: 1 leader_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3ac07af5e5a409d93de7a09ab8476bd" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 39727 } health_report { overall_health: HEALTHY } } }
W20260806 22:41:22.493222 10778 tablet.cc:2369] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:22.505407 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.503136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49752 (local address 127.16.235.65:39727)
0806 22:41:22.503202 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:22.503204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:22.503213 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:41:22.503256 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:22.503257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:22.503290 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:22.503357 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:41:22.503494 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.503968 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.504677 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:22.504871 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:22.504925 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:22.505066 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:41:22.505173 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:41:22.505237 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:22.505272 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 22:41:22.505291 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:41:22.507506 10520 raft_consensus.cc:1261] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [term 1 FOLLOWER]: Refusing update from remote peer f3ac07af5e5a409d93de7a09ab8476bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:22.507503 10706 raft_consensus.cc:1261] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [term 1 FOLLOWER]: Refusing update from remote peer f3ac07af5e5a409d93de7a09ab8476bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:22.507941 10810 consensus_queue.cc:1035] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cd90ac26c124c44a66258644682fe74" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 38267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:22.508090 10806 consensus_queue.cc:1035] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "484db560bab64a859b234b7c43f2e13f" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 39681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:22.510762 10813 mvcc.cc:205] Tried to move back new op lower bound from 7315685713947803648 to 7315685713885437952. Current Snapshot: MvccSnapshot[applied={T|T < 7315685713947803648}]
I20260806 22:41:22.513528 10121 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:22.513630 10121 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:22.514524 10121 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9448238377d49d3a568278254fc1f50: Sending DeleteTablet for 3 replicas of tablet 5f645dcbc2e0471fb2798f2d4c634c78
I20260806 22:41:22.514967 10308 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f645dcbc2e0471fb2798f2d4c634c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:22 UTC) from {username='slave'} at 127.0.0.1:49750
I20260806 22:41:22.515013 10674 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f645dcbc2e0471fb2798f2d4c634c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:22 UTC) from {username='slave'} at 127.0.0.1:43514
I20260806 22:41:22.515244 10492 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f645dcbc2e0471fb2798f2d4c634c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:22 UTC) from {username='slave'} at 127.0.0.1:39862
I20260806 22:41:22.515337 10828 tablet_replica.cc:330] T 5f645dcbc2e0471fb2798f2d4c634c78 P f3ac07af5e5a409d93de7a09ab8476bd: stopping tablet replica
I20260806 22:41:22.515491 10830 tablet_replica.cc:330] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: stopping tablet replica
I20260806 22:41:22.515612 10830 raft_consensus.cc:2229] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:41:22.515636 10832 tablet_replica.cc:330] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: stopping tablet replica
I20260806 22:41:22.515722 10832 raft_consensus.cc:2229] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:41:22.515725 10830 raft_consensus.cc:2258] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:22.515718 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 10213
I20260806 22:41:22.515826 10832 raft_consensus.cc:2258] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:22.521163 10832 ts_tablet_manager.cc:1905] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:22.521311 10830 ts_tablet_manager.cc:1905] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:22.523010 10832 ts_tablet_manager.cc:1918] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:22.523006 10830 ts_tablet_manager.cc:1918] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:22.523105 10832 log.cc:1196] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/5f645dcbc2e0471fb2798f2d4c634c78
I20260806 22:41:22.523106 10830 log.cc:1196] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/5f645dcbc2e0471fb2798f2d4c634c78
I20260806 22:41:22.523411 10830 ts_tablet_manager.cc:1939] T 5f645dcbc2e0471fb2798f2d4c634c78 P 3cd90ac26c124c44a66258644682fe74: Deleting consensus metadata
I20260806 22:41:22.523434 10832 ts_tablet_manager.cc:1939] T 5f645dcbc2e0471fb2798f2d4c634c78 P 484db560bab64a859b234b7c43f2e13f: Deleting consensus metadata
I20260806 22:41:22.523916 10102 catalog_manager.cc:4841] TS 3cd90ac26c124c44a66258644682fe74 (127.16.235.67:38267): tablet 5f645dcbc2e0471fb2798f2d4c634c78 (table test-table [id=b3383dcc52ae46bb8216402c15e873ee]) successfully deleted
I20260806 22:41:22.523919 10103 catalog_manager.cc:4841] TS 484db560bab64a859b234b7c43f2e13f (127.16.235.66:39681): tablet 5f645dcbc2e0471fb2798f2d4c634c78 (table test-table [id=b3383dcc52ae46bb8216402c15e873ee]) successfully deleted
W20260806 22:41:22.525898 10103 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:39727 (error 108)
I20260806 22:41:22.526212 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 10405
I20260806 22:41:22.528613 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:22.526315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:39727 (local address 127.0.0.1:49766)
0806 22:41:22.528506 (+ 2191us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:22.528565 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:39727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2164}
W20260806 22:41:22.530886 10103 catalog_manager.cc:4584] TS f3ac07af5e5a409d93de7a09ab8476bd (127.16.235.65:39727): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f645dcbc2e0471fb2798f2d4c634c78: Network error: Client connection negotiation failed: client connection to 127.16.235.65:39727: connect: Connection refused (error 111)
I20260806 22:41:22.535573 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 10593
I20260806 22:41:22.545874 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 10078
2026-08-06T22:41:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (915 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:22 dist-test-slave-3kk6 krb5kdc[10837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:22 dist-test-slave-3kk6 krb5kdc[10837](info): set up 2 sockets
Aug 06 22:41:22 dist-test-slave-3kk6 krb5kdc[10837](info): commencing operation
krb5kdc: starting...
W20260806 22:41:23.779814 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:23 dist-test-slave-3kk6 krb5kdc[10837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:23.913422 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34937
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:34937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:24.000826 10855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:24.001103 10855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:24.001145 10855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:24.002346 10855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:24.002427 10855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:24.002452 10855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:24.002475 10855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:24.002506 10855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:24.004102 10855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:34937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:24.004410 10855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:24.004657 10855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:24.009910 10866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:24.009881 10870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:24.009891 10863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:24.010142 10855 server_base.cc:1034] running on GCE node
I20260806 22:41:24.010493 10855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:24.010885 10855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:24.012042 10855 hybrid_clock.cc:648] HybridClock initialized: now 1786056084012018 us; error 33 us; skew 500 ppm
Aug 06 22:41:24 dist-test-slave-3kk6 krb5kdc[10837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056084, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:24.014986 10855 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:24.015440 10855 webserver.cc:460] Webserver started at http://127.16.235.126:35967/ using document root <none> and password file <none>
I20260806 22:41:24.015651 10855 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:24.015695 10855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:24.015798 10855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:24.016908 10855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "9182804b03be4fa6acf9e746c15f6f6b"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "2659eb100e0ecb8ff0f77fba64417135"
server_key_iv: "c9bcc401f654be7c9a216eedba20b7ad"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.017246 10855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "9182804b03be4fa6acf9e746c15f6f6b"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "2659eb100e0ecb8ff0f77fba64417135"
server_key_iv: "c9bcc401f654be7c9a216eedba20b7ad"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.018637 10855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:41:24.019526 10880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.019737 10855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:24.019817 10855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "9182804b03be4fa6acf9e746c15f6f6b"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "2659eb100e0ecb8ff0f77fba64417135"
server_key_iv: "c9bcc401f654be7c9a216eedba20b7ad"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.019877 10855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:24.040791 10855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:24.043458 10855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:24.043591 10855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:24.047909 10855 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:34937
I20260806 22:41:24.047999 10947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:34937 every 8 connection(s)
I20260806 22:41:24.048382 10855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:24.049367 10948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:24.049944 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 10855
I20260806 22:41:24.050078 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:24.050269 17325 external_mini_cluster.cc:1428] Setting key 0c73c13a2424e1a5dadd55904e6b5b1f
I20260806 22:41:24.052561 10948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b: Bootstrap starting.
I20260806 22:41:24.053218 10948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:24.053532 10948 log.cc:824] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:24.054365 10948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b: No bootstrap required, opened a new log
I20260806 22:41:24.057075 10948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9182804b03be4fa6acf9e746c15f6f6b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34937 } }
I20260806 22:41:24.057236 10948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:24.057260 10948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9182804b03be4fa6acf9e746c15f6f6b, State: Initialized, Role: FOLLOWER
I20260806 22:41:24.057401 10948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9182804b03be4fa6acf9e746c15f6f6b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34937 } }
I20260806 22:41:24.057480 10948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:24.057535 10948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:24.057585 10948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [term 0 FOLLOWER]: Advancing to term 1
Aug 06 22:41:24 dist-test-slave-3kk6 krb5kdc[10837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:24.058279 10948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9182804b03be4fa6acf9e746c15f6f6b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34937 } }
I20260806 22:41:24.058400 10948 leader_election.cc:302] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9182804b03be4fa6acf9e746c15f6f6b; no voters:
I20260806 22:41:24.058606 10948 leader_election.cc:288] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:24.058635 10958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:24.058827 10958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [term 1 LEADER]: Becoming Leader. State: Replica: 9182804b03be4fa6acf9e746c15f6f6b, State: Running, Role: LEADER
I20260806 22:41:24.058955 10948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:24.058990 10958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9182804b03be4fa6acf9e746c15f6f6b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34937 } }
I20260806 22:41:24.059413 10962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9182804b03be4fa6acf9e746c15f6f6b. Latest consensus state: current_term: 1 leader_uuid: "9182804b03be4fa6acf9e746c15f6f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9182804b03be4fa6acf9e746c15f6f6b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34937 } } }
I20260806 22:41:24.059434 10959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9182804b03be4fa6acf9e746c15f6f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9182804b03be4fa6acf9e746c15f6f6b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34937 } } }
I20260806 22:41:24.059617 10959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:24.059849 10962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:24.060196 10973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:24.060833 10973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:24.062434 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.050896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44290 (local address 127.16.235.126:34937)
0806 22:41:24.051101 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.051109 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.051136 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:41:24.051556 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.051562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.051609 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.051749 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:24.052609 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.053644 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.054734 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.055148 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.055239 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.058202 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:24.058207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:24.058207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:24.058221 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:24.060595 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.061041 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.061045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.061045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.061188 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.061428 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.061429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.061429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.061708 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:24.061813 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.062085 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.062211 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
I20260806 22:41:24.062812 10973 catalog_manager.cc:1347] Generated new cluster ID: 572444cc9be04069846748d516e3bd29
I20260806 22:41:24.062866 10973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:24.087972 10973 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:24.088569 10973 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:24.092672 10973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b: Generated new TSK 0
I20260806 22:41:24.092904 10973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:41:24.132202 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34937
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:24.219300 10991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:24.219637 10991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:24.219682 10991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:24.220947 10991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:24.221002 10991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:24.221031 10991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:41:24.222537 10991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:24.222831 10991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:24.223048 10991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:24.228459 10998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:24.228598 10991 server_base.cc:1034] running on GCE node
W20260806 22:41:24.228456 11001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:24.228848 11003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:24.229103 10991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:24.229349 10991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:24.230495 10991 hybrid_clock.cc:648] HybridClock initialized: now 1786056084230483 us; error 30 us; skew 500 ppm
Aug 06 22:41:24 dist-test-slave-3kk6 krb5kdc[10837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056084, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:24.233435 10991 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:41:24.233873 10991 webserver.cc:460] Webserver started at http://127.16.235.65:36375/ using document root <none> and password file <none>
I20260806 22:41:24.234135 10991 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:24.234220 10991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:24.234385 10991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:24.235579 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "d1780add883945d980bf4047650c21c6"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "03d3199ea166ebf2af9a6f1da4bd2f26"
server_key_iv: "cd01ce7585e9bad49fbe11651b82ee19"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.235965 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "d1780add883945d980bf4047650c21c6"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "03d3199ea166ebf2af9a6f1da4bd2f26"
server_key_iv: "cd01ce7585e9bad49fbe11651b82ee19"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.237389 10991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:41:24.238168 11012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.238500 10991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:24.238598 10991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "d1780add883945d980bf4047650c21c6"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "03d3199ea166ebf2af9a6f1da4bd2f26"
server_key_iv: "cd01ce7585e9bad49fbe11651b82ee19"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.238699 10991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:24.250985 10991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:24.253201 10991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:24.253337 10991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:24.253548 10991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:24.253898 10991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:24.253933 10991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.253957 10991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:24.254002 10991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.260430 10991 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:46161
I20260806 22:41:24.260491 11188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:46161 every 8 connection(s)
I20260806 22:41:24.260859 10991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:41:24 dist-test-slave-3kk6 krb5kdc[10837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056084, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:24.268990 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 10991
I20260806 22:41:24.269150 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:41:24.269419 17325 external_mini_cluster.cc:1428] Setting key 29f933b48b4cc1d885b045378e97050c
I20260806 22:41:24.271350 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.261753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:56761 (local address 127.16.235.126:34937)
0806 22:41:24.261830 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.261832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.262045 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:41:24.262548 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.262549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.262559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.262624 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:24.263729 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.264245 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.265337 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.265532 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.265586 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.268075 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:24.268077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:24.268077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:24.268085 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:24.269733 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.270207 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.270209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.270209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.270277 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.270572 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.270573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.270573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.270773 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:24.270857 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.271153 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.271257 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:41:24.271574 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.261672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34937 (local address 127.16.235.65:56761)
0806 22:41:24.261891 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:24.261946 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:24.262461 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:24.262801 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:24.262805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:24.263412 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:24.263624 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.263629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.264333 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.264334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.265254 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.265257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.265609 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.265609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.265756 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.266384 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:24.266394 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:24.267968 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:24.269821 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:24.269829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:24.269831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:24.270127 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:24.270374 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:24.270376 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:24.270376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:24.270502 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:24.270859 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:24.270868 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:24.271015 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 22:41:24.271131 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:41:24.271314 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":118,"threads_started":1}
I20260806 22:41:24.272588 11189 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34937
I20260806 22:41:24.272718 11189 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:24.272907 11189 heartbeater.cc:511] Master 127.16.235.126:34937 requested a full tablet report, sending...
I20260806 22:41:24.273543 10905 ts_manager.cc:194] Registered new tserver with Master: d1780add883945d980bf4047650c21c6 (127.16.235.65:46161)
I20260806 22:41:24.274482 10905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:56761
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:41:24.297818 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34937
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:24.383097 11194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:24.383339 11194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:24.383379 11194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:24.384509 11194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:24.384583 11194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:24.384634 11194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:41:24.386133 11194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:24.386412 11194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:24.386709 11194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:24.392062 11205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:24.392138 11201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:24.392103 11203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:24.392251 11194 server_base.cc:1034] running on GCE node
I20260806 22:41:24.392539 11194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:24.392812 11194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:24.393982 11194 hybrid_clock.cc:648] HybridClock initialized: now 1786056084393969 us; error 46 us; skew 500 ppm
Aug 06 22:41:24 dist-test-slave-3kk6 krb5kdc[10837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056084, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:24.396658 11194 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:41:24.397122 11194 webserver.cc:460] Webserver started at http://127.16.235.66:35853/ using document root <none> and password file <none>
I20260806 22:41:24.397373 11194 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:24.397420 11194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:24.397599 11194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:24.398787 11194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "5202678df971421a9bd04fe07bb25989"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "b088af261c4862605cc94c74728a12f5"
server_key_iv: "401aed2b1f7927b64e2767ca332c14df"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.399173 11194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "5202678df971421a9bd04fe07bb25989"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "b088af261c4862605cc94c74728a12f5"
server_key_iv: "401aed2b1f7927b64e2767ca332c14df"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.400722 11194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:41:24.401561 11214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.401809 11194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:24.401914 11194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "5202678df971421a9bd04fe07bb25989"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "b088af261c4862605cc94c74728a12f5"
server_key_iv: "401aed2b1f7927b64e2767ca332c14df"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.401998 11194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:24.427099 11194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:24.429234 11194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:24.429412 11194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:24.429694 11194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:24.430058 11194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:24.430101 11194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.430125 11194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:24.430177 11194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.436493 11194 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:46621
I20260806 22:41:24.436573 11390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:46621 every 8 connection(s)
I20260806 22:41:24.436949 11194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:41:24 dist-test-slave-3kk6 krb5kdc[10837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056084, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:24.444144 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 11194
I20260806 22:41:24.444279 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:41:24.444502 17325 external_mini_cluster.cc:1428] Setting key 9aa2850c3662484a76e3665e58a038df
I20260806 22:41:24.447408 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.437805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:33751 (local address 127.16.235.126:34937)
0806 22:41:24.437878 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.437881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.438128 (+ 247us) server_negotiation.cc:407] Connection header received
0806 22:41:24.438631 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.438632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.438644 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.438740 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:24.439744 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.440272 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.441307 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.441590 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.441678 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.444246 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:24.444247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:24.444247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:24.444256 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:24.445863 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.446310 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.446311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.446312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.446377 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.446675 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.446676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.446677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.446806 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:24.446886 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.447264 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.447341 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:41:24.447695 11397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.437788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34937 (local address 127.16.235.66:33751)
0806 22:41:24.437986 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:24.438027 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:24.438536 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:24.438826 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:24.438830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:24.439435 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:24.439641 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.439645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.440360 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.440361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.441212 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.441214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.441691 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.441693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.441858 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.442476 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:24.442485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:24.444141 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:24.445955 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:24.445963 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:24.445966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:24.446205 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:24.446449 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:24.446450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:24.446450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:24.446573 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:24.446877 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:24.446884 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:24.447076 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 22:41:24.447214 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 22:41:24.447415 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":109,"threads_started":1}
I20260806 22:41:24.448382 11394 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34937
I20260806 22:41:24.448516 11394 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:24.448707 11394 heartbeater.cc:511] Master 127.16.235.126:34937 requested a full tablet report, sending...
I20260806 22:41:24.449139 10905 ts_manager.cc:194] Registered new tserver with Master: 5202678df971421a9bd04fe07bb25989 (127.16.235.66:46621)
I20260806 22:41:24.449542 10905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:33751
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:41:24.474028 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34937
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:24.560498 11401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:24.560743 11401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:24.560770 11401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:24.561978 11401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:24.562031 11401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:24.562086 11401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:41:24.563879 11401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:24.564138 11401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:24.564399 11401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:24.569738 11407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:24.569710 11411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:24.569748 11408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:24.569922 11401 server_base.cc:1034] running on GCE node
I20260806 22:41:24.570252 11401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:24.570538 11401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:24.571730 11401 hybrid_clock.cc:648] HybridClock initialized: now 1786056084571736 us; error 49 us; skew 500 ppm
Aug 06 22:41:24 dist-test-slave-3kk6 krb5kdc[10837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056084, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:24.574437 11401 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:41:24.575008 11401 webserver.cc:460] Webserver started at http://127.16.235.67:39135/ using document root <none> and password file <none>
I20260806 22:41:24.575261 11401 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:24.575336 11401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:24.575505 11401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:24.577008 11401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "f8bbe9b69ee7417c9a333a0d498f75f0"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "3799d4cb2dc5b000cb4b344849c17782"
server_key_iv: "7f3fccac16810bd8d722e0db9ecb03c7"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.577411 11401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "f8bbe9b69ee7417c9a333a0d498f75f0"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "3799d4cb2dc5b000cb4b344849c17782"
server_key_iv: "7f3fccac16810bd8d722e0db9ecb03c7"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.578954 11401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:41:24.580127 11420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.580346 11401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:24.580463 11401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "f8bbe9b69ee7417c9a333a0d498f75f0"
format_stamp: "Formatted at 2026-08-06 22:41:24 on dist-test-slave-3kk6"
server_key: "3799d4cb2dc5b000cb4b344849c17782"
server_key_iv: "7f3fccac16810bd8d722e0db9ecb03c7"
server_key_version: "encryptionkey@0"
I20260806 22:41:24.580543 11401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:24.611055 11401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:24.613138 11401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:24.613335 11401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:24.613641 11401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:24.613994 11401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:24.614053 11401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.614097 11401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:24.614125 11401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:24.620328 11401 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:32869
I20260806 22:41:24.620424 11586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:32869 every 8 connection(s)
I20260806 22:41:24.620777 11401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:41:24 dist-test-slave-3kk6 krb5kdc[10837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056084, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:24.630129 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.621651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:34851 (local address 127.16.235.126:34937)
0806 22:41:24.621732 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.621734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.621963 (+ 229us) server_negotiation.cc:407] Connection header received
0806 22:41:24.622464 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.622465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.622475 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.622533 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:24.623630 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.624189 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.625192 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.625385 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.625443 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.627713 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:24.627714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:24.627714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:24.627720 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:24.628955 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.629333 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.629333 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.629334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.629384 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.629599 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.629599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.629600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.629701 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:24.629752 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.629999 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.630061 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:41:24.630331 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.621632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34937 (local address 127.16.235.67:34851)
0806 22:41:24.621830 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:24.621871 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:24.622376 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:24.622615 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:24.622619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:24.623321 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:24.623527 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.623532 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.624281 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.624282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.625118 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.625121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.625464 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.625465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.625569 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.626150 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:24.626159 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:24.627627 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:24.629032 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:24.629039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:24.629041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:24.629267 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:24.629443 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:24.629443 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:24.629444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:24.629545 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:24.629759 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:24.629763 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:24.629886 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 22:41:24.629970 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:41:24.630096 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260806 22:41:24.630707 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 11401
I20260806 22:41:24.630843 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:41:24.630957 11587 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34937
I20260806 22:41:24.631060 17325 external_mini_cluster.cc:1428] Setting key 1db3fee107ef9a2ae1611e6263eb5da8
I20260806 22:41:24.631067 11587 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:24.631269 11587 heartbeater.cc:511] Master 127.16.235.126:34937 requested a full tablet report, sending...
I20260806 22:41:24.631762 10905 ts_manager.cc:194] Registered new tserver with Master: f8bbe9b69ee7417c9a333a0d498f75f0 (127.16.235.67:32869)
I20260806 22:41:24.632251 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:41:24.632339 10905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:34851
I20260806 22:41:24.640573 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.633846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44300 (local address 127.16.235.126:34937)
0806 22:41:24.633923 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.633925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.634038 (+ 113us) server_negotiation.cc:407] Connection header received
0806 22:41:24.634115 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.634116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.634125 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.634170 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:24.634817 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.635304 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.636046 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.636258 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.636336 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.637790 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:24.637791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:24.637791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:24.637799 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:24.639298 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.639708 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.639710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.639710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.639775 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:24.640009 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:24.640010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:24.640011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:24.640157 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:24.640286 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.640402 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.640469 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:41:24.642356 10905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:41:24.644069 10905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:41:24.650938 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.646812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.16.235.67:32869)
0806 22:41:24.647062 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.647066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.647073 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:41:24.647106 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.647108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.647121 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.647223 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:24.647503 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.648390 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.649801 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.650459 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.650549 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.650615 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.650806 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.650850 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":102,"threads_started":1}
I20260806 22:41:24.650925 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.646507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:32869 (local address 127.0.0.1:42456)
0806 22:41:24.646733 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:24.646788 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:24.646862 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:24.647272 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:24.647275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:24.647278 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:24.647356 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.647358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.648500 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.648501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.649707 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.649710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.650550 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.650551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.650635 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.650676 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:41:24.650783 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:41:24.650846 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":72,"threads_started":1}
I20260806 22:41:24.651260 11601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.646507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:46621 (local address 127.0.0.1:56962)
0806 22:41:24.647216 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:24.647226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:24.647334 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:24.647611 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:24.647612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:24.647614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:24.647656 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.647657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.648905 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.648906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.649950 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.649952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.651003 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.651005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.651069 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.651073 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:24.651134 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:41:24.651177 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":28,"thread_start_us":96,"threads_started":1}
I20260806 22:41:24.651316 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.646784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56962 (local address 127.16.235.66:46621)
0806 22:41:24.646994 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.647000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.647301 (+ 301us) server_negotiation.cc:407] Connection header received
0806 22:41:24.647513 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.647516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.647532 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.647615 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:24.647714 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.648792 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.650039 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.650828 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.650905 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.650963 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.651174 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.651236 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
I20260806 22:41:24.651368 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.647014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.16.235.65:46161)
0806 22:41:24.647177 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.647182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.647547 (+ 365us) server_negotiation.cc:407] Connection header received
0806 22:41:24.647596 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.647599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.647615 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.647683 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:24.647897 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.648934 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.650157 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.650900 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.650960 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.651008 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.651224 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.651269 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260806 22:41:24.651533 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.646522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:46161 (local address 127.0.0.1:40252)
0806 22:41:24.647434 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:24.647443 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:24.647488 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:24.647765 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:24.647766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:24.647768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:24.647809 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.647810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.649051 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.649053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.650069 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.650072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.651038 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.651038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.651092 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.651096 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:24.651145 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:41:24.651179 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":118,"threads_started":1}
I20260806 22:41:24.652277 11080 tablet_service.cc:1467] Processing CreateTablet for tablet c18b747229ad4b42aaad5efeecb1cddc (DEFAULT_TABLE table=test-table [id=72a4dfe09ca14c21a7313f30d4190ed4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:24.652268 11282 tablet_service.cc:1467] Processing CreateTablet for tablet c18b747229ad4b42aaad5efeecb1cddc (DEFAULT_TABLE table=test-table [id=72a4dfe09ca14c21a7313f30d4190ed4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:24.652379 11492 tablet_service.cc:1467] Processing CreateTablet for tablet c18b747229ad4b42aaad5efeecb1cddc (DEFAULT_TABLE table=test-table [id=72a4dfe09ca14c21a7313f30d4190ed4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:24.652683 11080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c18b747229ad4b42aaad5efeecb1cddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:24.652683 11282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c18b747229ad4b42aaad5efeecb1cddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:24.652743 11492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c18b747229ad4b42aaad5efeecb1cddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:24.655578 11609 tablet_bootstrap.cc:492] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: Bootstrap starting.
I20260806 22:41:24.656093 11611 tablet_bootstrap.cc:492] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6: Bootstrap starting.
I20260806 22:41:24.656323 11612 tablet_bootstrap.cc:492] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: Bootstrap starting.
I20260806 22:41:24.656402 11609 tablet_bootstrap.cc:654] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:24.656983 11609 log.cc:824] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:24.657060 11612 tablet_bootstrap.cc:654] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:24.657603 11612 log.cc:824] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:24.657863 11609 tablet_bootstrap.cc:492] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: No bootstrap required, opened a new log
I20260806 22:41:24.657945 11609 ts_tablet_manager.cc:1397] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:41:24.658423 11612 tablet_bootstrap.cc:492] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: No bootstrap required, opened a new log
I20260806 22:41:24.658496 11612 ts_tablet_manager.cc:1397] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:41:24.658499 11611 tablet_bootstrap.cc:654] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:24.659142 11611 log.cc:824] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:24.659999 11611 tablet_bootstrap.cc:492] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6: No bootstrap required, opened a new log
I20260806 22:41:24.660125 11611 ts_tablet_manager.cc:1397] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:41:24.660326 11609 raft_consensus.cc:348] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } }
I20260806 22:41:24.660545 11609 raft_consensus.cc:374] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:24.660435 11612 raft_consensus.cc:348] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } }
I20260806 22:41:24.660624 11609 raft_consensus.cc:729] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8bbe9b69ee7417c9a333a0d498f75f0, State: Initialized, Role: FOLLOWER
I20260806 22:41:24.660660 11612 raft_consensus.cc:374] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:24.660704 11612 raft_consensus.cc:729] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5202678df971421a9bd04fe07bb25989, State: Initialized, Role: FOLLOWER
I20260806 22:41:24.660753 11609 consensus_queue.cc:260] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } }
I20260806 22:41:24.660837 11612 consensus_queue.cc:260] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } }
I20260806 22:41:24.661056 11609 ts_tablet_manager.cc:1428] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:41:24.661159 11612 ts_tablet_manager.cc:1428] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:41:24.661188 11394 heartbeater.cc:503] Master 127.16.235.126:34937 was elected leader, sending a full tablet report...
I20260806 22:41:24.661249 11587 heartbeater.cc:503] Master 127.16.235.126:34937 was elected leader, sending a full tablet report...
I20260806 22:41:24.662539 11611 raft_consensus.cc:348] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } }
I20260806 22:41:24.662796 11611 raft_consensus.cc:374] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:24.662870 11611 raft_consensus.cc:729] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1780add883945d980bf4047650c21c6, State: Initialized, Role: FOLLOWER
I20260806 22:41:24.662989 11611 consensus_queue.cc:260] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } }
I20260806 22:41:24.663268 11611 ts_tablet_manager.cc:1428] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:41:24.663326 11189 heartbeater.cc:503] Master 127.16.235.126:34937 was elected leader, sending a full tablet report...
W20260806 22:41:24.687573 11395 tablet.cc:2369] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:41:24.761665 11190 tablet.cc:2369] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:24.818862 11622 raft_consensus.cc:482] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:24.819157 11622 raft_consensus.cc:504] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } }
I20260806 22:41:24.819581 11622 leader_election.cc:288] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8bbe9b69ee7417c9a333a0d498f75f0 (127.16.235.67:32869), 5202678df971421a9bd04fe07bb25989 (127.16.235.66:46621)
I20260806 22:41:24.822727 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.819917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:42969 (local address 127.16.235.66:46621)
0806 22:41:24.819992 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.819994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.820081 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:41:24.820140 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.820141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.820153 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.820243 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:24.820430 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.820920 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.821906 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.822296 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.822389 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.822399 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.822583 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.822625 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:41:24.822738 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.820030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:53097 (local address 127.16.235.67:32869)
0806 22:41:24.820078 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.820080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.820088 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:41:24.820212 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.820217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.820226 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.820292 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:24.820493 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.820964 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.821925 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.822315 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.822408 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.822416 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.822583 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.822622 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:41:24.822784 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.819829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:32869 (local address 127.16.235.65:53097)
0806 22:41:24.820013 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:24.820026 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:24.820107 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:24.820297 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:24.820298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:24.820300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:24.820409 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.820411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.821064 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.821065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.821829 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.821831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.822408 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.822409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.822519 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.822524 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:41:24.822578 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:41:24.822621 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260806 22:41:24.822798 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.819827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:46621 (local address 127.16.235.65:42969)
0806 22:41:24.819986 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:24.819999 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:24.820073 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:24.820233 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:24.820234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:24.820237 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:24.820342 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.820344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.821012 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.821014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.821793 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:24.821796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.822389 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.822390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:24.822476 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.822480 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:24.822566 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:41:24.822611 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":5,"thread_start_us":94,"threads_started":1}
I20260806 22:41:24.823211 11516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18b747229ad4b42aaad5efeecb1cddc" candidate_uuid: "d1780add883945d980bf4047650c21c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" is_pre_election: true
I20260806 22:41:24.823419 11516 raft_consensus.cc:2454] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1780add883945d980bf4047650c21c6 in term 0.
I20260806 22:41:24.823395 11318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18b747229ad4b42aaad5efeecb1cddc" candidate_uuid: "d1780add883945d980bf4047650c21c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5202678df971421a9bd04fe07bb25989" is_pre_election: true
I20260806 22:41:24.823567 11318 raft_consensus.cc:2454] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1780add883945d980bf4047650c21c6 in term 0.
I20260806 22:41:24.823760 11014 leader_election.cc:302] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1780add883945d980bf4047650c21c6, f8bbe9b69ee7417c9a333a0d498f75f0; no voters:
I20260806 22:41:24.823942 11622 raft_consensus.cc:2781] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:41:24.824005 11622 raft_consensus.cc:482] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:41:24.824023 11622 raft_consensus.cc:3037] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:24.824829 11622 raft_consensus.cc:504] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } }
I20260806 22:41:24.824981 11622 leader_election.cc:288] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [CANDIDATE]: Term 1 election: Requested vote from peers f8bbe9b69ee7417c9a333a0d498f75f0 (127.16.235.67:32869), 5202678df971421a9bd04fe07bb25989 (127.16.235.66:46621)
I20260806 22:41:24.825146 11318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18b747229ad4b42aaad5efeecb1cddc" candidate_uuid: "d1780add883945d980bf4047650c21c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5202678df971421a9bd04fe07bb25989"
I20260806 22:41:24.825153 11516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18b747229ad4b42aaad5efeecb1cddc" candidate_uuid: "d1780add883945d980bf4047650c21c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0"
I20260806 22:41:24.825234 11516 raft_consensus.cc:3037] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:24.825225 11318 raft_consensus.cc:3037] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:24.826002 11516 raft_consensus.cc:2454] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1780add883945d980bf4047650c21c6 in term 1.
I20260806 22:41:24.826005 11318 raft_consensus.cc:2454] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1780add883945d980bf4047650c21c6 in term 1.
I20260806 22:41:24.826222 11015 leader_election.cc:302] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5202678df971421a9bd04fe07bb25989, d1780add883945d980bf4047650c21c6; no voters:
I20260806 22:41:24.826327 11622 raft_consensus.cc:2781] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:24.826508 11622 raft_consensus.cc:686] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [term 1 LEADER]: Becoming Leader. State: Replica: d1780add883945d980bf4047650c21c6, State: Running, Role: LEADER
I20260806 22:41:24.826623 11622 consensus_queue.cc:237] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } }
I20260806 22:41:24.827551 10905 catalog_manager.cc:5495] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1780add883945d980bf4047650c21c6 (127.16.235.65). New cstate: current_term: 1 leader_uuid: "d1780add883945d980bf4047650c21c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1780add883945d980bf4047650c21c6" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46161 } health_report { overall_health: HEALTHY } } }
I20260806 22:41:24.833323 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.830784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40262 (local address 127.16.235.65:46161)
0806 22:41:24.830828 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:24.830830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:24.830912 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:41:24.830972 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:24.830972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:24.830991 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:24.831057 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:41:24.831283 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.831786 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.832571 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:24.832798 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:24.832871 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:24.833001 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:41:24.833087 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:41:24.833162 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:24.833210 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 22:41:24.833218 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:41:24.835276 11516 raft_consensus.cc:1261] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [term 1 FOLLOWER]: Refusing update from remote peer d1780add883945d980bf4047650c21c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:24.835289 11318 raft_consensus.cc:1261] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [term 1 FOLLOWER]: Refusing update from remote peer d1780add883945d980bf4047650c21c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:24.835603 11622 consensus_queue.cc:1035] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8bbe9b69ee7417c9a333a0d498f75f0" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 32869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:24.835757 11627 consensus_queue.cc:1035] T c18b747229ad4b42aaad5efeecb1cddc P d1780add883945d980bf4047650c21c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5202678df971421a9bd04fe07bb25989" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:24.838581 11632 mvcc.cc:205] Tried to move back new op lower bound from 7315685723482320896 to 7315685723450667008. Current Snapshot: MvccSnapshot[applied={T|T < 7315685723482320896}]
I20260806 22:41:24.841149 10904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:24.841256 10904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:24.842082 10904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9182804b03be4fa6acf9e746c15f6f6b: Sending DeleteTablet for 3 replicas of tablet c18b747229ad4b42aaad5efeecb1cddc
I20260806 22:41:24.842450 11282 tablet_service.cc:1514] Processing DeleteTablet for tablet c18b747229ad4b42aaad5efeecb1cddc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:24 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:56962
I20260806 22:41:24.842450 11080 tablet_service.cc:1514] Processing DeleteTablet for tablet c18b747229ad4b42aaad5efeecb1cddc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:24 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:40252
I20260806 22:41:24.842515 11492 tablet_service.cc:1514] Processing DeleteTablet for tablet c18b747229ad4b42aaad5efeecb1cddc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:24 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:42456
I20260806 22:41:24.842797 11647 tablet_replica.cc:330] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: stopping tablet replica
I20260806 22:41:24.842936 11647 raft_consensus.cc:2229] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:41:24.842994 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 10991
I20260806 22:41:24.843091 11647 raft_consensus.cc:2258] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:24.851703 11647 ts_tablet_manager.cc:1905] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:24.853926 11647 ts_tablet_manager.cc:1918] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:24.854035 11647 log.cc:1196] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/c18b747229ad4b42aaad5efeecb1cddc
I20260806 22:41:24.854364 11647 ts_tablet_manager.cc:1939] T c18b747229ad4b42aaad5efeecb1cddc P f8bbe9b69ee7417c9a333a0d498f75f0: Deleting consensus metadata
I20260806 22:41:24.854934 10882 catalog_manager.cc:4841] TS f8bbe9b69ee7417c9a333a0d498f75f0 (127.16.235.67:32869): tablet c18b747229ad4b42aaad5efeecb1cddc (table test-table [id=72a4dfe09ca14c21a7313f30d4190ed4]) successfully deleted
I20260806 22:41:24.855460 11644 tablet_replica.cc:330] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: stopping tablet replica
I20260806 22:41:24.855619 11644 raft_consensus.cc:2229] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:41:24.855757 11644 raft_consensus.cc:2258] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:24.856316 11644 ts_tablet_manager.cc:1905] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:24.858294 11644 ts_tablet_manager.cc:1918] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:24.858434 11644 log.cc:1196] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/c18b747229ad4b42aaad5efeecb1cddc
I20260806 22:41:24.858765 11644 ts_tablet_manager.cc:1939] T c18b747229ad4b42aaad5efeecb1cddc P 5202678df971421a9bd04fe07bb25989: Deleting consensus metadata
I20260806 22:41:24.859261 10884 catalog_manager.cc:4841] TS 5202678df971421a9bd04fe07bb25989 (127.16.235.66:46621): tablet c18b747229ad4b42aaad5efeecb1cddc (table test-table [id=72a4dfe09ca14c21a7313f30d4190ed4]) successfully deleted
W20260806 22:41:24.860767 10885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:46161 (error 108)
I20260806 22:41:24.861070 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 11194
I20260806 22:41:24.861330 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:24.861192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:46161 (local address 127.0.0.1:40274)
0806 22:41:24.861251 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:24.861287 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:41:24.861562 10885 catalog_manager.cc:4584] TS d1780add883945d980bf4047650c21c6 (127.16.235.65:46161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c18b747229ad4b42aaad5efeecb1cddc: Network error: Client connection negotiation failed: client connection to 127.16.235.65:46161: connect: Connection refused (error 111)
I20260806 22:41:24.872954 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 11401
I20260806 22:41:24.884114 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 10855
2026-08-06T22:41:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2339 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T22:41:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:24Z Disabled control of system clock
I20260806 22:41:24.921209 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:46793
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:46793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:41:25.008787 11653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:25.009052 11653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:25.009075 11653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:25.010248 11653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:25.010290 11653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:25.010303 11653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:25.010317 11653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:25.010329 11653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:25.011838 11653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:46793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:46793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:25.012076 11653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:25.012353 11653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:25.017601 11660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:25.017604 11665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:25.017639 11659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:25.017975 11653 server_base.cc:1034] running on GCE node
I20260806 22:41:25.018246 11653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:25.018582 11653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:25.019762 11653 hybrid_clock.cc:648] HybridClock initialized: now 1786056085019739 us; error 43 us; skew 500 ppm
I20260806 22:41:25.021198 11653 webserver.cc:460] Webserver started at http://127.16.235.126:44035/ using document root <none> and password file <none>
I20260806 22:41:25.021481 11653 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:25.021554 11653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:25.021714 11653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:25.022925 11653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "fb6581b40c79420eb60f7f94a62eb444"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "61965359b8fdd99a40172ca1c5a15f17"
server_key_iv: "84f187486e0291e80e1a451d6078765a"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.023306 11653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "fb6581b40c79420eb60f7f94a62eb444"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "61965359b8fdd99a40172ca1c5a15f17"
server_key_iv: "84f187486e0291e80e1a451d6078765a"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.024784 11653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:41:25.025532 11676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.025720 11653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:41:25.025833 11653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "fb6581b40c79420eb60f7f94a62eb444"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "61965359b8fdd99a40172ca1c5a15f17"
server_key_iv: "84f187486e0291e80e1a451d6078765a"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.025915 11653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:25.033775 11653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:25.034106 11653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:25.034272 11653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:25.038362 11653 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:46793
I20260806 22:41:25.038398 11753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:46793 every 8 connection(s)
I20260806 22:41:25.038825 11653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:25.039880 11754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:25.042250 11754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444: Bootstrap starting.
I20260806 22:41:25.042922 11754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:25.043180 11754 log.cc:824] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:25.043897 11754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444: No bootstrap required, opened a new log
I20260806 22:41:25.045555 11754 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6581b40c79420eb60f7f94a62eb444" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46793 } }
I20260806 22:41:25.045707 11754 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:25.045732 11754 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6581b40c79420eb60f7f94a62eb444, State: Initialized, Role: FOLLOWER
I20260806 22:41:25.045849 11754 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6581b40c79420eb60f7f94a62eb444" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46793 } }
I20260806 22:41:25.045938 11754 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:25.045979 11754 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:25.046032 11754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:25.046619 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 11653
I20260806 22:41:25.046764 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:25.046792 11754 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6581b40c79420eb60f7f94a62eb444" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46793 } }
I20260806 22:41:25.046907 11754 leader_election.cc:302] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb6581b40c79420eb60f7f94a62eb444; no voters:
I20260806 22:41:25.047003 17325 external_mini_cluster.cc:1428] Setting key 4bbc797392d7f3b06a3d068bef8b753d
I20260806 22:41:25.047111 11754 leader_election.cc:288] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:25.047175 11760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:25.047355 11760 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [term 1 LEADER]: Becoming Leader. State: Replica: fb6581b40c79420eb60f7f94a62eb444, State: Running, Role: LEADER
I20260806 22:41:25.047574 11760 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6581b40c79420eb60f7f94a62eb444" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46793 } }
I20260806 22:41:25.047399 11754 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:25.048089 11762 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb6581b40c79420eb60f7f94a62eb444. Latest consensus state: current_term: 1 leader_uuid: "fb6581b40c79420eb60f7f94a62eb444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6581b40c79420eb60f7f94a62eb444" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46793 } } }
I20260806 22:41:25.048179 11763 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb6581b40c79420eb60f7f94a62eb444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb6581b40c79420eb60f7f94a62eb444" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 46793 } } }
I20260806 22:41:25.048290 11763 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:25.048282 11762 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:25.048595 11775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:25.049180 11775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:25.050791 11775 catalog_manager.cc:1347] Generated new cluster ID: 65d0e8598e4c42219052817a58e01013
I20260806 22:41:25.050855 11775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:25.053568 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.047630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58822 (local address 127.16.235.126:46793)
0806 22:41:25.047936 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.047946 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.047979 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:41:25.048114 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.048121 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.048193 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.048668 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:25.048798 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.050040 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.051153 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.051857 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.052017 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.052794 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:25.052798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:25.052799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:25.053027 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:25.053072 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:25.053088 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:25.053168 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.053250 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.053335 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":86,"threads_started":1}
I20260806 22:41:25.055819 11775 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:25.056499 11775 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:25.059836 11775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444: Generated new TSK 0
I20260806 22:41:25.060034 11775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:25.062255 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46793
--builtin_ntp_servers=127.16.235.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:41:25.179008 11791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:25.179251 11791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:25.179275 11791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:25.180521 11791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:25.180572 11791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:25.180599 11791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:41:25.182067 11791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:25.182336 11791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:25.182720 11791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:25.188064 11797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:25.188053 11798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:25.188057 11800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:25.188396 11791 server_base.cc:1034] running on GCE node
I20260806 22:41:25.188617 11791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:25.188908 11791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:25.190099 11791 hybrid_clock.cc:648] HybridClock initialized: now 1786056085190070 us; error 48 us; skew 500 ppm
I20260806 22:41:25.191748 11791 webserver.cc:460] Webserver started at http://127.16.235.65:34371/ using document root <none> and password file <none>
I20260806 22:41:25.192011 11791 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:25.192087 11791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:25.192251 11791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:25.193425 11791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "1fa5f14cceed48f5a7eb3127f7373150"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "9cf44a657147b548d4db60e4e67efea5"
server_key_iv: "cedc16daf558ac41bb40d9e41776f728"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.193831 11791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "1fa5f14cceed48f5a7eb3127f7373150"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "9cf44a657147b548d4db60e4e67efea5"
server_key_iv: "cedc16daf558ac41bb40d9e41776f728"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.195396 11791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:41:25.196309 11807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.196511 11791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:25.196627 11791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "1fa5f14cceed48f5a7eb3127f7373150"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "9cf44a657147b548d4db60e4e67efea5"
server_key_iv: "cedc16daf558ac41bb40d9e41776f728"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.196744 11791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:25.225595 11791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:25.226009 11791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:25.226189 11791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:25.226490 11791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:25.226950 11791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:25.227010 11791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.227059 11791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:25.227084 11791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.233843 11791 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:33131
I20260806 22:41:25.233893 11977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:33131 every 8 connection(s)
I20260806 22:41:25.234267 11791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:41:25.239843 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.235103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:59299 (local address 127.16.235.126:46793)
0806 22:41:25.235183 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.235185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.235458 (+ 273us) server_negotiation.cc:407] Connection header received
0806 22:41:25.235971 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.235972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.235985 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.236053 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:25.236750 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.237265 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.238299 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.238496 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.238566 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.239579 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:25.239580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:25.239580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:25.239587 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:25.239602 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:25.239606 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:25.239655 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.239748 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.239778 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:41:25.240018 11980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.235083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:46793 (local address 127.16.235.65:59299)
0806 22:41:25.235312 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:25.235357 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:25.235883 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:25.236141 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:25.236145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:25.236154 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:25.236672 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.236676 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.237367 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.237368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.238191 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.238194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.238588 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.238589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.238813 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.239435 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:41:25.239435 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:25.239464 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:41:25.239489 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:25.239669 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:25.239674 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:25.239677 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:25.239731 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:41:25.239813 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":120,"threads_started":1}
I20260806 22:41:25.240609 11978 heartbeater.cc:347] Connected to a master server at 127.16.235.126:46793
I20260806 22:41:25.240751 11978 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:25.240967 11978 heartbeater.cc:511] Master 127.16.235.126:46793 requested a full tablet report, sending...
I20260806 22:41:25.241526 11699 ts_manager.cc:194] Registered new tserver with Master: 1fa5f14cceed48f5a7eb3127f7373150 (127.16.235.65:33131)
I20260806 22:41:25.242194 11699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.65:59299
I20260806 22:41:25.242401 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 11791
I20260806 22:41:25.242538 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:41:25.242794 17325 external_mini_cluster.cc:1428] Setting key b6de604f5b6d9f62fef14acecc54d48f
I20260806 22:41:25.243819 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46793
--builtin_ntp_servers=127.16.235.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:41:25.329054 11981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:25.329289 11981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:25.329312 11981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:25.330529 11981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:25.330579 11981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:25.330606 11981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:41:25.332032 11981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:25.332250 11981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:25.332526 11981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:25.337838 11994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:25.337803 11989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:25.337857 11992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:25.337917 11981 server_base.cc:1034] running on GCE node
I20260806 22:41:25.338279 11981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:25.338583 11981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:25.339738 11981 hybrid_clock.cc:648] HybridClock initialized: now 1786056085339714 us; error 39 us; skew 500 ppm
I20260806 22:41:25.341104 11981 webserver.cc:460] Webserver started at http://127.16.235.66:42029/ using document root <none> and password file <none>
I20260806 22:41:25.341368 11981 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:25.341416 11981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:25.341596 11981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:25.342823 11981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "6dc7320d003543c1b436758fb8c006dd"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "dd8283d790d6d88de087ab6ec0e7424f"
server_key_iv: "788f58480411b87423bdc238b1fffeb3"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.343209 11981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "6dc7320d003543c1b436758fb8c006dd"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "dd8283d790d6d88de087ab6ec0e7424f"
server_key_iv: "788f58480411b87423bdc238b1fffeb3"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.344576 11981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:41:25.345310 12001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.345500 11981 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:41:25.345598 11981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "6dc7320d003543c1b436758fb8c006dd"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "dd8283d790d6d88de087ab6ec0e7424f"
server_key_iv: "788f58480411b87423bdc238b1fffeb3"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.345676 11981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:25.354359 11981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:25.354750 11981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:25.354912 11981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:25.355165 11981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:25.355546 11981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:25.355605 11981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.355649 11981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:25.355677 11981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.361874 11981 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:43811
I20260806 22:41:25.361918 12161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:43811 every 8 connection(s)
I20260806 22:41:25.362325 11981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:41:25.368072 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.363328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:56941 (local address 127.16.235.126:46793)
0806 22:41:25.363404 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.363406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.363707 (+ 301us) server_negotiation.cc:407] Connection header received
0806 22:41:25.364224 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.364225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.364236 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.364295 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:25.365017 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.365511 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.366544 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.366827 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.366884 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.367861 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:25.367861 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:25.367862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:25.367866 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:25.367877 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:25.367881 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:25.367919 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.367984 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.368007 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:41:25.368279 12166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.363327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:46793 (local address 127.16.235.66:56941)
0806 22:41:25.363557 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:25.363594 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:25.364131 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:25.364366 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:25.364369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:25.364378 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:25.364941 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.364946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.365605 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.365606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.366459 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.366461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.366913 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.366914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.367045 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.367702 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:41:25.367703 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:25.367757 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:41:25.367782 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:25.367922 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:25.367925 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:25.367927 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:41:25.367976 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:41:25.368060 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":99,"threads_started":1}
I20260806 22:41:25.368837 12162 heartbeater.cc:347] Connected to a master server at 127.16.235.126:46793
I20260806 22:41:25.368959 12162 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:25.369154 12162 heartbeater.cc:511] Master 127.16.235.126:46793 requested a full tablet report, sending...
I20260806 22:41:25.369604 11699 ts_manager.cc:194] Registered new tserver with Master: 6dc7320d003543c1b436758fb8c006dd (127.16.235.66:43811)
I20260806 22:41:25.369889 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 11981
I20260806 22:41:25.370007 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:41:25.370141 11699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.66:56941
I20260806 22:41:25.370255 17325 external_mini_cluster.cc:1428] Setting key f7a8a9fdbafcf2a7caad8144eacd6865
I20260806 22:41:25.371309 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46793
--builtin_ntp_servers=127.16.235.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:41:25.457810 12167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:25.458034 12167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:25.458056 12167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:25.459201 12167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:25.459237 12167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:25.459262 12167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:41:25.460541 12167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:25.460737 12167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:25.461001 12167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:25.466091 12175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:25.466114 12173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:25.466101 12179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:25.466325 12167 server_base.cc:1034] running on GCE node
I20260806 22:41:25.466554 12167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:25.466845 12167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:25.468037 12167 hybrid_clock.cc:648] HybridClock initialized: now 1786056085468023 us; error 33 us; skew 500 ppm
I20260806 22:41:25.469326 12167 webserver.cc:460] Webserver started at http://127.16.235.67:33083/ using document root <none> and password file <none>
I20260806 22:41:25.469561 12167 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:25.469604 12167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:25.469707 12167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:25.470886 12167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "0214a73f226a47279758bc02bcd746c9"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "61ed2d425581d05c637ae4e4c1863e16"
server_key_iv: "b106be20f723e9a3cc20bd18f2e766f2"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.471271 12167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "0214a73f226a47279758bc02bcd746c9"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "61ed2d425581d05c637ae4e4c1863e16"
server_key_iv: "b106be20f723e9a3cc20bd18f2e766f2"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.472709 12167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:41:25.473469 12186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.473670 12167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:25.473752 12167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "0214a73f226a47279758bc02bcd746c9"
format_stamp: "Formatted at 2026-08-06 22:41:25 on dist-test-slave-3kk6"
server_key: "61ed2d425581d05c637ae4e4c1863e16"
server_key_iv: "b106be20f723e9a3cc20bd18f2e766f2"
server_key_version: "encryptionkey@0"
I20260806 22:41:25.473809 12167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:25.494235 12167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:25.494558 12167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:25.494704 12167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:25.494932 12167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:25.495257 12167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:25.495292 12167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.495316 12167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:25.495329 12167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:25.501549 12167 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:43267
I20260806 22:41:25.501659 12341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:43267 every 8 connection(s)
I20260806 22:41:25.502004 12167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:41:25.507720 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 12167
I20260806 22:41:25.507870 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:41:25.507876 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.502930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:53509 (local address 127.16.235.126:46793)
0806 22:41:25.502979 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.502982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.503217 (+ 235us) server_negotiation.cc:407] Connection header received
0806 22:41:25.503852 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.503853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.503869 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.503925 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:25.504664 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.505191 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.506237 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.506469 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.506553 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.507554 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:25.507556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:25.507556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:25.507562 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:25.507585 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:25.507589 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:25.507649 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.507756 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.507794 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:41:25.508116 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.502889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:46793 (local address 127.16.235.67:53509)
0806 22:41:25.503079 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:25.503118 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:25.503697 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:25.504001 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:25.504005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:25.504015 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:25.504555 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.504560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.505314 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.505315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.506133 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.506135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.506568 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.506569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.506779 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.507377 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:41:25.507378 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:25.507407 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:41:25.507432 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:25.507651 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:25.507658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:25.507662 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:25.507758 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:41:25.507881 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260806 22:41:25.508124 17325 external_mini_cluster.cc:1428] Setting key 4bc707687fabfa764950ceceebac143c
I20260806 22:41:25.508685 12342 heartbeater.cc:347] Connected to a master server at 127.16.235.126:46793
I20260806 22:41:25.508831 12342 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:25.509068 12342 heartbeater.cc:511] Master 127.16.235.126:46793 requested a full tablet report, sending...
I20260806 22:41:25.509454 11698 ts_manager.cc:194] Registered new tserver with Master: 0214a73f226a47279758bc02bcd746c9 (127.16.235.67:43267)
I20260806 22:41:25.509795 11698 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.67:53509
I20260806 22:41:25.520219 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:41:25.524873 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.521785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58824 (local address 127.16.235.126:46793)
0806 22:41:25.521864 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.521867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.522013 (+ 146us) server_negotiation.cc:407] Connection header received
0806 22:41:25.522067 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.522068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.522082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.522123 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:25.522273 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.522836 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.523580 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.523811 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.523867 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.524618 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:25.524619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:25.524619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:25.524623 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:25.524634 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:25.524637 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:25.524700 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.524744 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.524753 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:41:25.526845 11698 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:41:25.528434 11698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:41:25.535032 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.530808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48816 (local address 127.16.235.67:43267)
0806 22:41:25.531026 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.531032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.531320 (+ 288us) server_negotiation.cc:407] Connection header received
0806 22:41:25.531411 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.531413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.531428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.531502 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:25.531732 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.532513 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.533880 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.534579 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.534688 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.534793 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.534895 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.534935 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":128,"threads_started":1}
I20260806 22:41:25.535017 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.530693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:43267 (local address 127.0.0.1:48816)
0806 22:41:25.531224 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:25.531234 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:25.531325 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:25.531576 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:25.531577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:25.531579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:25.531622 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.531624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.532592 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.532593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.533790 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.533793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.534695 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.534696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.534784 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.534797 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:41:25.534877 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:41:25.534933 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260806 22:41:25.535205 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.531055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.16.235.65:33131)
0806 22:41:25.531275 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.531279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.531289 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:41:25.531333 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.531335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.531347 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.531436 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:25.531682 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.532841 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.534093 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.534595 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.534726 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.534826 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.535063 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.535110 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":97,"threads_started":1}
I20260806 22:41:25.535485 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.530650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:33131 (local address 127.0.0.1:41618)
0806 22:41:25.531050 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:25.531093 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:25.531164 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:25.531489 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:25.531492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:25.531495 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:25.531570 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.531573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.532938 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.532939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.533996 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.533998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.534862 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.534863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.534916 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.534919 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:25.534965 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:41:25.534996 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260806 22:41:25.535748 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.530893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:43811 (local address 127.0.0.1:51228)
0806 22:41:25.531531 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:25.531541 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:25.531594 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:25.531834 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:25.531835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:25.531837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:25.531870 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.531871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.533070 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.533071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.534081 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.534084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.535122 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.535123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.535571 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.535578 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:41:25.535637 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:41:25.535681 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":27,"thread_start_us":82,"threads_started":1}
I20260806 22:41:25.536136 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.531003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.16.235.66:43811)
0806 22:41:25.531193 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.531197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.531680 (+ 483us) server_negotiation.cc:407] Connection header received
0806 22:41:25.531741 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.531744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.531759 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.531837 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:25.531937 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.532978 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.534178 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.534929 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.534996 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.535080 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.536006 (+ 926us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.536053 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":116,"threads_started":1}
I20260806 22:41:25.536062 12255 tablet_service.cc:1467] Processing CreateTablet for tablet 2eaa809795ec49a794b1278bc7931fa1 (DEFAULT_TABLE table=test-table [id=bb2b4bde953340ab8486b65c342e3f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:25.536237 11882 tablet_service.cc:1467] Processing CreateTablet for tablet 2eaa809795ec49a794b1278bc7931fa1 (DEFAULT_TABLE table=test-table [id=bb2b4bde953340ab8486b65c342e3f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:25.536479 12255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eaa809795ec49a794b1278bc7931fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:25.536614 11882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eaa809795ec49a794b1278bc7931fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:25.539371 12361 tablet_bootstrap.cc:492] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: Bootstrap starting.
I20260806 22:41:25.539654 12362 tablet_bootstrap.cc:492] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150: Bootstrap starting.
I20260806 22:41:25.539767 12070 tablet_service.cc:1467] Processing CreateTablet for tablet 2eaa809795ec49a794b1278bc7931fa1 (DEFAULT_TABLE table=test-table [id=bb2b4bde953340ab8486b65c342e3f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:25.540143 12070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eaa809795ec49a794b1278bc7931fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:25.540171 12361 tablet_bootstrap.cc:654] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:25.540343 12362 tablet_bootstrap.cc:654] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:25.540524 12361 log.cc:824] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:25.540666 12362 log.cc:824] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:25.541308 12361 tablet_bootstrap.cc:492] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: No bootstrap required, opened a new log
I20260806 22:41:25.541397 12361 ts_tablet_manager.cc:1397] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:41:25.541486 12362 tablet_bootstrap.cc:492] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150: No bootstrap required, opened a new log
I20260806 22:41:25.541548 12362 ts_tablet_manager.cc:1397] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:41:25.543490 12361 raft_consensus.cc:348] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } }
I20260806 22:41:25.543665 12361 raft_consensus.cc:374] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:25.543706 12361 raft_consensus.cc:729] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0214a73f226a47279758bc02bcd746c9, State: Initialized, Role: FOLLOWER
I20260806 22:41:25.543723 12366 tablet_bootstrap.cc:492] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: Bootstrap starting.
I20260806 22:41:25.543828 12361 consensus_queue.cc:260] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } }
I20260806 22:41:25.544123 12361 ts_tablet_manager.cc:1428] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:41:25.544165 12342 heartbeater.cc:503] Master 127.16.235.126:46793 was elected leader, sending a full tablet report...
I20260806 22:41:25.544253 12362 raft_consensus.cc:348] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } }
I20260806 22:41:25.544394 12366 tablet_bootstrap.cc:654] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:25.544445 12362 raft_consensus.cc:374] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:25.544517 12362 raft_consensus.cc:729] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa5f14cceed48f5a7eb3127f7373150, State: Initialized, Role: FOLLOWER
I20260806 22:41:25.544662 12362 consensus_queue.cc:260] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } }
I20260806 22:41:25.544703 12366 log.cc:824] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:25.544929 12362 ts_tablet_manager.cc:1428] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:41:25.544974 11978 heartbeater.cc:503] Master 127.16.235.126:46793 was elected leader, sending a full tablet report...
I20260806 22:41:25.545800 12366 tablet_bootstrap.cc:492] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: No bootstrap required, opened a new log
I20260806 22:41:25.545887 12366 ts_tablet_manager.cc:1397] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:41:25.548017 12366 raft_consensus.cc:348] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } }
I20260806 22:41:25.548163 12366 raft_consensus.cc:374] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:25.548213 12366 raft_consensus.cc:729] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc7320d003543c1b436758fb8c006dd, State: Initialized, Role: FOLLOWER
I20260806 22:41:25.548338 12366 consensus_queue.cc:260] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } }
I20260806 22:41:25.548663 12366 ts_tablet_manager.cc:1428] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:41:25.548725 12162 heartbeater.cc:503] Master 127.16.235.126:46793 was elected leader, sending a full tablet report...
W20260806 22:41:25.612984 12164 tablet.cc:2369] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:41:25.735057 11979 tablet.cc:2369] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:41:25.752648 12343 tablet.cc:2369] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:25.870512 12374 raft_consensus.cc:482] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:25.870764 12374 raft_consensus.cc:504] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } }
I20260806 22:41:25.871213 12374 leader_election.cc:288] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fa5f14cceed48f5a7eb3127f7373150 (127.16.235.65:33131), 0214a73f226a47279758bc02bcd746c9 (127.16.235.67:43267)
I20260806 22:41:25.874250 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.871532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:52189 (local address 127.16.235.65:33131)
0806 22:41:25.871597 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.871600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.871677 (+ 77us) server_negotiation.cc:407] Connection header received
0806 22:41:25.871761 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.871764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.871778 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.871845 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:25.872017 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.872510 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.873504 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.873859 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.873973 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.873984 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.874138 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.874179 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:41:25.874274 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.871438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:33131 (local address 127.16.235.66:52189)
0806 22:41:25.871578 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:25.871593 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:25.871684 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:25.871863 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:25.871864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:25.871866 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:25.871944 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.871946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.872608 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.872609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.873397 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.873399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.873980 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.873982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.874048 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.874052 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:25.874129 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:41:25.874172 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260806 22:41:25.874344 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.871616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:48823 (local address 127.16.235.67:43267)
0806 22:41:25.871729 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.871732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.871794 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:41:25.871887 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.871888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.871897 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.871978 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:25.872131 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.872597 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.873557 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.873947 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.874037 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.874047 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.874235 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.874271 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 22:41:25.874393 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.871438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:43267 (local address 127.16.235.66:48823)
0806 22:41:25.871716 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:25.871729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:25.871815 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:25.871982 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:25.871983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:25.871985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:25.872056 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.872058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.872717 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.872718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.873457 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:25.873459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.874052 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.874053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:25.874132 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.874135 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:25.874225 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:41:25.874268 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260806 22:41:25.874754 11913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eaa809795ec49a794b1278bc7931fa1" candidate_uuid: "6dc7320d003543c1b436758fb8c006dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa5f14cceed48f5a7eb3127f7373150" is_pre_election: true
I20260806 22:41:25.874759 12283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eaa809795ec49a794b1278bc7931fa1" candidate_uuid: "6dc7320d003543c1b436758fb8c006dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0214a73f226a47279758bc02bcd746c9" is_pre_election: true
I20260806 22:41:25.874931 12283 raft_consensus.cc:2454] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc7320d003543c1b436758fb8c006dd in term 0.
I20260806 22:41:25.874931 11913 raft_consensus.cc:2454] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dc7320d003543c1b436758fb8c006dd in term 0.
I20260806 22:41:25.875192 12008 leader_election.cc:302] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0214a73f226a47279758bc02bcd746c9, 6dc7320d003543c1b436758fb8c006dd; no voters:
I20260806 22:41:25.875337 12374 raft_consensus.cc:2781] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:41:25.875396 12374 raft_consensus.cc:482] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:41:25.875418 12374 raft_consensus.cc:3037] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:25.876259 12374 raft_consensus.cc:504] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } }
I20260806 22:41:25.876415 12374 leader_election.cc:288] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [CANDIDATE]: Term 1 election: Requested vote from peers 1fa5f14cceed48f5a7eb3127f7373150 (127.16.235.65:33131), 0214a73f226a47279758bc02bcd746c9 (127.16.235.67:43267)
I20260806 22:41:25.876585 11913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eaa809795ec49a794b1278bc7931fa1" candidate_uuid: "6dc7320d003543c1b436758fb8c006dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa5f14cceed48f5a7eb3127f7373150"
I20260806 22:41:25.876608 12283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eaa809795ec49a794b1278bc7931fa1" candidate_uuid: "6dc7320d003543c1b436758fb8c006dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0214a73f226a47279758bc02bcd746c9"
I20260806 22:41:25.876690 12283 raft_consensus.cc:3037] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:25.876690 11913 raft_consensus.cc:3037] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:25.877697 11913 raft_consensus.cc:2454] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc7320d003543c1b436758fb8c006dd in term 1.
I20260806 22:41:25.877697 12283 raft_consensus.cc:2454] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dc7320d003543c1b436758fb8c006dd in term 1.
I20260806 22:41:25.877942 12005 leader_election.cc:302] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0214a73f226a47279758bc02bcd746c9, 1fa5f14cceed48f5a7eb3127f7373150, 6dc7320d003543c1b436758fb8c006dd; no voters:
I20260806 22:41:25.878034 12374 raft_consensus.cc:2781] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:25.878249 12374 raft_consensus.cc:686] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 1 LEADER]: Becoming Leader. State: Replica: 6dc7320d003543c1b436758fb8c006dd, State: Running, Role: LEADER
I20260806 22:41:25.878367 12374 consensus_queue.cc:237] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } }
I20260806 22:41:25.879271 11699 catalog_manager.cc:5495] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dc7320d003543c1b436758fb8c006dd (127.16.235.66). New cstate: current_term: 1 leader_uuid: "6dc7320d003543c1b436758fb8c006dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dc7320d003543c1b436758fb8c006dd" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 43811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 } health_report { overall_health: UNKNOWN } } }
I20260806 22:41:25.889430 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.887028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.16.235.66:43811)
0806 22:41:25.887100 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:25.887102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:25.887110 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:41:25.887149 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:25.887150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:25.887175 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:25.887268 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:41:25.887411 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.887886 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.888660 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:25.888872 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:25.888948 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:25.889093 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:41:25.889194 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:41:25.889302 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:25.889335 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:41:25.889346 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:41:25.891557 11913 raft_consensus.cc:1261] T 2eaa809795ec49a794b1278bc7931fa1 P 1fa5f14cceed48f5a7eb3127f7373150 [term 1 FOLLOWER]: Refusing update from remote peer 6dc7320d003543c1b436758fb8c006dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:25.891557 12283 raft_consensus.cc:1261] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [term 1 FOLLOWER]: Refusing update from remote peer 6dc7320d003543c1b436758fb8c006dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:25.891985 12378 consensus_queue.cc:1035] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0214a73f226a47279758bc02bcd746c9" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:25.892167 12374 consensus_queue.cc:1035] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fa5f14cceed48f5a7eb3127f7373150" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 33131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:25.896262 12386 mvcc.cc:205] Tried to move back new op lower bound from 7315685727808454656 to 7315685727758434304. Current Snapshot: MvccSnapshot[applied={T|T < 7315685727808454656 or (T in {7315685727808454656})}]
I20260806 22:41:25.897414 11699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:25.897537 11699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:25.898344 11699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb6581b40c79420eb60f7f94a62eb444: Sending DeleteTablet for 3 replicas of tablet 2eaa809795ec49a794b1278bc7931fa1
I20260806 22:41:25.898759 12070 tablet_service.cc:1514] Processing DeleteTablet for tablet 2eaa809795ec49a794b1278bc7931fa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:25 UTC) from {username='slave'} at 127.0.0.1:51228
I20260806 22:41:25.898753 12255 tablet_service.cc:1514] Processing DeleteTablet for tablet 2eaa809795ec49a794b1278bc7931fa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:25 UTC) from {username='slave'} at 127.0.0.1:48816
I20260806 22:41:25.898820 11882 tablet_service.cc:1514] Processing DeleteTablet for tablet 2eaa809795ec49a794b1278bc7931fa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:25 UTC) from {username='slave'} at 127.0.0.1:41618
I20260806 22:41:25.899096 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 11791
I20260806 22:41:25.912606 12395 tablet_replica.cc:330] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: stopping tablet replica
I20260806 22:41:25.912827 12395 raft_consensus.cc:2229] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:41:25.912981 12395 raft_consensus.cc:2258] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:25.914827 12394 tablet_replica.cc:330] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: stopping tablet replica
I20260806 22:41:25.915002 12394 raft_consensus.cc:2229] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:41:25.915148 12394 raft_consensus.cc:2258] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:25.916587 12394 ts_tablet_manager.cc:1905] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:25.916769 12395 ts_tablet_manager.cc:1905] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:25.918641 12395 ts_tablet_manager.cc:1918] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:25.918852 12395 log.cc:1196] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/2eaa809795ec49a794b1278bc7931fa1
I20260806 22:41:25.918851 12394 ts_tablet_manager.cc:1918] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:25.918918 12394 log.cc:1196] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/2eaa809795ec49a794b1278bc7931fa1
W20260806 22:41:25.918991 11679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:33131 (error 108)
I20260806 22:41:25.919232 12394 ts_tablet_manager.cc:1939] T 2eaa809795ec49a794b1278bc7931fa1 P 6dc7320d003543c1b436758fb8c006dd: Deleting consensus metadata
I20260806 22:41:25.919232 12395 ts_tablet_manager.cc:1939] T 2eaa809795ec49a794b1278bc7931fa1 P 0214a73f226a47279758bc02bcd746c9: Deleting consensus metadata
I20260806 22:41:25.919680 11679 catalog_manager.cc:4841] TS 6dc7320d003543c1b436758fb8c006dd (127.16.235.66:43811): tablet 2eaa809795ec49a794b1278bc7931fa1 (table test-table [id=bb2b4bde953340ab8486b65c342e3f92]) successfully deleted
I20260806 22:41:25.919806 11681 catalog_manager.cc:4841] TS 0214a73f226a47279758bc02bcd746c9 (127.16.235.67:43267): tablet 2eaa809795ec49a794b1278bc7931fa1 (table test-table [id=bb2b4bde953340ab8486b65c342e3f92]) successfully deleted
I20260806 22:41:25.919976 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 11981
I20260806 22:41:25.923341 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:25.923186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:33131 (local address 127.0.0.1:41620)
0806 22:41:25.923248 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:25.923309 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:33131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:41:25.923535 11679 catalog_manager.cc:4584] TS 1fa5f14cceed48f5a7eb3127f7373150 (127.16.235.65:33131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2eaa809795ec49a794b1278bc7931fa1: Network error: Client connection negotiation failed: client connection to 127.16.235.65:33131: connect: Connection refused (error 111)
I20260806 22:41:25.927932 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 12167
I20260806 22:41:25.940689 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 11653
2026-08-06T22:41:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1054 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:26 dist-test-slave-3kk6 krb5kdc[12402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:26 dist-test-slave-3kk6 krb5kdc[12402](info): set up 2 sockets
Aug 06 22:41:26 dist-test-slave-3kk6 krb5kdc[12402](info): commencing operation
krb5kdc: starting...
W20260806 22:41:27.173116 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:27 dist-test-slave-3kk6 krb5kdc[12402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:27.309855 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33693
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:33693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:27.396346 12417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:27.396644 12417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:27.396690 12417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:27.397974 12417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:27.398053 12417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:27.398077 12417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:27.398097 12417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:27.398131 12417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:27.399731 12417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:33693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:27.400028 12417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:27.400279 12417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:27.405571 12429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:27.405598 12433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:27.405663 12431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:27.405711 12417 server_base.cc:1034] running on GCE node
I20260806 22:41:27.406091 12417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:27.406441 12417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:27.407606 12417 hybrid_clock.cc:648] HybridClock initialized: now 1786056087407599 us; error 39 us; skew 500 ppm
Aug 06 22:41:27 dist-test-slave-3kk6 krb5kdc[12402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056087, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:27.410324 12417 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:27.410902 12417 webserver.cc:460] Webserver started at http://127.16.235.126:36857/ using document root <none> and password file <none>
I20260806 22:41:27.411103 12417 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:27.411150 12417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:27.411250 12417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:27.412353 12417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "ac6a01a9543d4061818a6b5a2d039cc6"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "5e925b8c3bde3fde18d1bd2e3ab91762"
server_key_iv: "71dbfa4ec974f54566b953c2816be4be"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.412698 12417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "ac6a01a9543d4061818a6b5a2d039cc6"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "5e925b8c3bde3fde18d1bd2e3ab91762"
server_key_iv: "71dbfa4ec974f54566b953c2816be4be"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.414076 12417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:27.414906 12441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.415107 12417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:27.415225 12417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "ac6a01a9543d4061818a6b5a2d039cc6"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "5e925b8c3bde3fde18d1bd2e3ab91762"
server_key_iv: "71dbfa4ec974f54566b953c2816be4be"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.415305 12417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:27.451812 12417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:27.454413 12417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:27.454622 12417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:27.459374 12417 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:33693
I20260806 22:41:27.459385 12508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:33693 every 8 connection(s)
I20260806 22:41:27.459854 12417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:27.460847 12509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:27.463366 12509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6: Bootstrap starting.
I20260806 22:41:27.464030 12509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:27.464390 12509 log.cc:824] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:27.465253 12509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6: No bootstrap required, opened a new log
I20260806 22:41:27.466439 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 12417
I20260806 22:41:27.466542 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:27.466825 17325 external_mini_cluster.cc:1428] Setting key 74b871a611f415f432fb970410933d48
I20260806 22:41:27.467352 12509 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6a01a9543d4061818a6b5a2d039cc6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33693 } }
I20260806 22:41:27.467593 12509 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:27.467682 12509 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6a01a9543d4061818a6b5a2d039cc6, State: Initialized, Role: FOLLOWER
I20260806 22:41:27.467842 12509 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6a01a9543d4061818a6b5a2d039cc6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33693 } }
I20260806 22:41:27.467939 12509 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:27.467991 12509 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:27.468045 12509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:27.469349 12509 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6a01a9543d4061818a6b5a2d039cc6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33693 } }
I20260806 22:41:27.469449 12509 leader_election.cc:302] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6a01a9543d4061818a6b5a2d039cc6; no voters:
I20260806 22:41:27.469648 12509 leader_election.cc:288] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:27.469691 12517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:27.469866 12517 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [term 1 LEADER]: Becoming Leader. State: Replica: ac6a01a9543d4061818a6b5a2d039cc6, State: Running, Role: LEADER
I20260806 22:41:27.469964 12509 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:27.470022 12517 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6a01a9543d4061818a6b5a2d039cc6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33693 } }
I20260806 22:41:27.470435 12518 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6a01a9543d4061818a6b5a2d039cc6. Latest consensus state: current_term: 1 leader_uuid: "ac6a01a9543d4061818a6b5a2d039cc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6a01a9543d4061818a6b5a2d039cc6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33693 } } }
I20260806 22:41:27.470539 12518 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:27.470839 12524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:27.471374 12524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:27.472546 12519 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac6a01a9543d4061818a6b5a2d039cc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6a01a9543d4061818a6b5a2d039cc6" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33693 } } }
I20260806 22:41:27.473104 12519 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:27.473268 12524 catalog_manager.cc:1347] Generated new cluster ID: 1e5975da7e0446c5a823e7ea85c2d610
I20260806 22:41:27.473330 12524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:41:27 dist-test-slave-3kk6 krb5kdc[12402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:27.479543 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:27.467479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.16.235.126:33693)
0806 22:41:27.467699 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:27.467710 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:27.467755 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:41:27.467893 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:27.467899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:27.467972 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:27.468126 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:27.469002 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.470263 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.471417 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.472063 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.472184 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:27.475380 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:27.475385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:27.475385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:27.475398 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:27.477453 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:27.477953 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:27.477956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:27.477956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:27.478099 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:27.478343 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:27.478344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:27.478344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:27.478703 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:27.478870 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:27.478991 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:41:27.479143 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":103,"threads_started":1}
I20260806 22:41:27.481925 12524 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:27.482466 12524 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:27.487936 12524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6: Generated new TSK 0
I20260806 22:41:27.488142 12524 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.235.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
Entry for principal kudu/127.16.235.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab.
I20260806 22:41:27.518256 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33693
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:27.606942 12547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:27.607205 12547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:27.607247 12547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:27.608448 12547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:27.608515 12547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:27.608560 12547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:41:27.610092 12547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.65.keytab
--principal=kudu/127.16.235.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:27.610378 12547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:27.610633 12547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:27.616988 12563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:27.617002 12556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:27.617002 12560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:27.617406 12547 server_base.cc:1034] running on GCE node
I20260806 22:41:27.617633 12547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:27.617899 12547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:27.619046 12547 hybrid_clock.cc:648] HybridClock initialized: now 1786056087619028 us; error 60 us; skew 500 ppm
Aug 06 22:41:27 dist-test-slave-3kk6 krb5kdc[12402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056087, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:27.621743 12547 init.cc:377] Logged in from keytab as kudu/127.16.235.65@KRBTEST.COM (short username kudu)
I20260806 22:41:27.622294 12547 webserver.cc:460] Webserver started at http://127.16.235.65:43773/ using document root <none> and password file <none>
I20260806 22:41:27.622555 12547 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:27.622627 12547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:27.622823 12547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:27.623944 12547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "fd9d487cba8c403d90fb23cd20766a40"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "43ab157ef7ec094aa33b27b0a8d36cfb"
server_key_iv: "125b7a186794529c6a681faaf7df3687"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.624399 12547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "fd9d487cba8c403d90fb23cd20766a40"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "43ab157ef7ec094aa33b27b0a8d36cfb"
server_key_iv: "125b7a186794529c6a681faaf7df3687"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.625856 12547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:27.626722 12575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.626933 12547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:27.627040 12547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "fd9d487cba8c403d90fb23cd20766a40"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "43ab157ef7ec094aa33b27b0a8d36cfb"
server_key_iv: "125b7a186794529c6a681faaf7df3687"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.627127 12547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:27.642613 12547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:27.644856 12547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:27.645040 12547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:27.645512 12547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:27.645934 12547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:27.645998 12547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.646034 12547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:27.646065 12547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.652757 12547 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:46691
I20260806 22:41:27.652849 12729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:46691 every 8 connection(s)
I20260806 22:41:27.653229 12547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:41:27.654678 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 12547
I20260806 22:41:27.654803 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:41:27.655045 17325 external_mini_cluster.cc:1428] Setting key 69813f54ddc6236089110d9a82f946d1
Aug 06 22:41:27 dist-test-slave-3kk6 krb5kdc[12402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056087, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.65@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:27.665086 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:27.654126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:36873 (local address 127.16.235.126:33693)
0806 22:41:27.654193 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:27.654196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:27.654420 (+ 224us) server_negotiation.cc:407] Connection header received
0806 22:41:27.655276 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:27.655278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:27.655297 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:27.655364 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:27.656835 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.657336 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.658541 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.658913 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.658992 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:27.661615 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:27.661617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:27.661617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:27.661630 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:27.663440 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:27.663894 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:27.663895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:27.663896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:27.663967 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:27.664230 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:27.664231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:27.664232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:27.664391 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:27.664471 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:27.664815 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 22:41:27.664942 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:41:27.665184 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:27.654105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33693 (local address 127.16.235.65:36873)
0806 22:41:27.654274 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:27.654315 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:27.655133 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:27.655537 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:27.655543 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:27.656416 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:27.656718 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:27.656725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.657440 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.657441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:27.658437 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:27.658440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.659003 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.659004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:27.659112 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:27.659726 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:27.659735 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:27.661498 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:27.663549 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:27.663557 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:27.663559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:27.663808 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:27.664044 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:27.664045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:27.664045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:27.664156 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:27.664498 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:27.664506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:27.664648 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 22:41:27.664759 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:41:27.664912 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.16.235.66@KRBTEST.COM; defaulting to no policy
I20260806 22:41:27.665788 12731 heartbeater.cc:347] Connected to a master server at 127.16.235.126:33693
I20260806 22:41:27.665907 12731 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:27.666122 12731 heartbeater.cc:511] Master 127.16.235.126:33693 requested a full tablet report, sending...
I20260806 22:41:27.666937 12464 ts_manager.cc:194] Registered new tserver with Master: fd9d487cba8c403d90fb23cd20766a40 (127.16.235.65:46691)
I20260806 22:41:27.667599 12464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.65@KRBTEST.COM'} at 127.16.235.65:36873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
Entry for principal kudu/127.16.235.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab.
I20260806 22:41:27.686143 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33693
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:27.772541 12738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:27.772765 12738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:27.772791 12738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:27.773960 12738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:27.774010 12738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:27.774039 12738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:41:27.775565 12738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.66.keytab
--principal=kudu/127.16.235.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:27.775825 12738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:27.776086 12738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:41:27.781572 12738 server_base.cc:1034] running on GCE node
W20260806 22:41:27.781541 12747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:27.781591 12744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:27.781550 12745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:27.781879 12738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:27.782173 12738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:27.783320 12738 hybrid_clock.cc:648] HybridClock initialized: now 1786056087783305 us; error 46 us; skew 500 ppm
Aug 06 22:41:27 dist-test-slave-3kk6 krb5kdc[12402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056087, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:27.785846 12738 init.cc:377] Logged in from keytab as kudu/127.16.235.66@KRBTEST.COM (short username kudu)
I20260806 22:41:27.786305 12738 webserver.cc:460] Webserver started at http://127.16.235.66:44251/ using document root <none> and password file <none>
I20260806 22:41:27.786521 12738 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:27.786562 12738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:27.786722 12738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:27.787774 12738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "a55e64996f954b8990fbb5d4413b0b8e"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "3f17e78063114e6e6eb80d3ef8c4eab8"
server_key_iv: "aa2507ecbd79233fa1f09d208962cbee"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.788087 12738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "a55e64996f954b8990fbb5d4413b0b8e"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "3f17e78063114e6e6eb80d3ef8c4eab8"
server_key_iv: "aa2507ecbd79233fa1f09d208962cbee"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.789397 12738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:41:27.790102 12761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.790285 12738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:41:27.790397 12738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "a55e64996f954b8990fbb5d4413b0b8e"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "3f17e78063114e6e6eb80d3ef8c4eab8"
server_key_iv: "aa2507ecbd79233fa1f09d208962cbee"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.790480 12738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:27.804870 12738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:27.807219 12738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:27.807420 12738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:27.807693 12738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:27.808054 12738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:27.808112 12738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.808157 12738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:27.808202 12738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.814858 12738 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:41705
I20260806 22:41:27.814903 12912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:41705 every 8 connection(s)
I20260806 22:41:27.815281 12738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:41:27 dist-test-slave-3kk6 krb5kdc[12402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056087, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.66@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:27.822827 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 12738
I20260806 22:41:27.822927 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:41:27.823206 17325 external_mini_cluster.cc:1428] Setting key 153dcdaa493b644444922714d2eec092
I20260806 22:41:27.826427 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:27.816111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:37293 (local address 127.16.235.126:33693)
0806 22:41:27.816189 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:27.816192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:27.816427 (+ 235us) server_negotiation.cc:407] Connection header received
0806 22:41:27.816940 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:27.816941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:27.816953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:27.817010 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:27.818071 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.818703 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.819750 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.819951 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.820013 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:27.822765 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:27.822767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:27.822767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:27.822776 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:27.824516 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:27.825108 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:27.825108 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:27.825109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:27.825164 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:27.825497 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:27.825498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:27.825498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:27.825651 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:27.825727 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:27.826157 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 22:41:27.826287 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:41:27.826546 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:27.816087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33693 (local address 127.16.235.66:37293)
0806 22:41:27.816285 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:27.816335 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:27.816849 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:27.817070 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:27.817073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:27.817756 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:27.817975 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:27.817980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.818814 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.818816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:27.819670 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:27.819672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:27.820070 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:27.820071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:27.820197 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:27.820821 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:27.820831 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:27.822546 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:27.824624 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:27.824634 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:27.824637 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:27.825016 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:27.825231 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:27.825232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:27.825233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:27.825401 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:27.825767 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:27.825775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:27.825969 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 22:41:27.826102 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 22:41:27.826320 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260806 22:41:27.827162 12914 heartbeater.cc:347] Connected to a master server at 127.16.235.126:33693
I20260806 22:41:27.827286 12914 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:27.827487 12914 heartbeater.cc:511] Master 127.16.235.126:33693 requested a full tablet report, sending...
I20260806 22:41:27.827955 12464 ts_manager.cc:194] Registered new tserver with Master: a55e64996f954b8990fbb5d4413b0b8e (127.16.235.66:41705)
I20260806 22:41:27.828508 12464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.66@KRBTEST.COM'} at 127.16.235.66:37293
WARNING: no policy specified for kudu/127.16.235.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
Entry for principal kudu/127.16.235.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab.
I20260806 22:41:27.854288 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33693
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:27.939879 12919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:27.940112 12919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:27.940140 12919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:27.941318 12919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:27.941368 12919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:27.941422 12919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:41:27.942970 12919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.67.keytab
--principal=kudu/127.16.235.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:33693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:27.943245 12919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:27.943511 12919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:27.948832 12927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:27.948864 12929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:27.948920 12925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:27.949138 12919 server_base.cc:1034] running on GCE node
I20260806 22:41:27.949374 12919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:27.949612 12919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:27.950730 12919 hybrid_clock.cc:648] HybridClock initialized: now 1786056087950707 us; error 40 us; skew 500 ppm
Aug 06 22:41:27 dist-test-slave-3kk6 krb5kdc[12402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056087, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:27.953380 12919 init.cc:377] Logged in from keytab as kudu/127.16.235.67@KRBTEST.COM (short username kudu)
I20260806 22:41:27.953893 12919 webserver.cc:460] Webserver started at http://127.16.235.67:39451/ using document root <none> and password file <none>
I20260806 22:41:27.954151 12919 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:27.954205 12919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:27.954391 12919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:27.955580 12919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "d8a7a4b96273441b8d5a50b97ddc52b7"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "42167a980ac2832d1556ffd5c0f0675a"
server_key_iv: "6d7594214300ac8c6acb0ce6f4f46a41"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.955998 12919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "d8a7a4b96273441b8d5a50b97ddc52b7"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "42167a980ac2832d1556ffd5c0f0675a"
server_key_iv: "6d7594214300ac8c6acb0ce6f4f46a41"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.957427 12919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:41:27.958290 12940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.958572 12919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:27.958712 12919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "d8a7a4b96273441b8d5a50b97ddc52b7"
format_stamp: "Formatted at 2026-08-06 22:41:27 on dist-test-slave-3kk6"
server_key: "42167a980ac2832d1556ffd5c0f0675a"
server_key_iv: "6d7594214300ac8c6acb0ce6f4f46a41"
server_key_version: "encryptionkey@0"
I20260806 22:41:27.958812 12919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:27.990113 12919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:27.992266 12919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:27.992466 12919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:27.992729 12919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:27.993088 12919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:27.993148 12919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.993193 12919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:27.993219 12919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:27.999774 12919 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:46583
I20260806 22:41:27.999831 13089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:46583 every 8 connection(s)
I20260806 22:41:28.000212 12919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:41:28.000619 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 12919
I20260806 22:41:28.000742 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:41:28.000953 17325 external_mini_cluster.cc:1428] Setting key 683c50b220e8a9073f7cd5ffeada4d70
Aug 06 22:41:28 dist-test-slave-3kk6 krb5kdc[12402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056087, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.67@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:28.010241 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.001090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:46305 (local address 127.16.235.126:33693)
0806 22:41:28.001153 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.001156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.001404 (+ 248us) server_negotiation.cc:407] Connection header received
0806 22:41:28.002143 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.002145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.002158 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.002201 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:28.003324 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.003839 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.004890 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.005100 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.005180 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.007649 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:28.007651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:28.007651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:28.007658 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:28.008978 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:28.009401 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:28.009402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:28.009402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:28.009457 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:28.009678 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:28.009679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:28.009679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:28.009788 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:28.009845 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.010091 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.010170 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:41:28.010439 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.001053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:33693 (local address 127.16.235.67:46305)
0806 22:41:28.001255 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.001308 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.002025 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.002280 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.002284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.002993 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:28.003197 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.003202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.003938 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.003938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.004787 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.004789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.005189 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.005190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.005325 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.005934 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:28.005943 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:28.007536 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:28.009058 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:28.009066 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:28.009068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:28.009332 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:28.009517 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:28.009518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:28.009518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:28.009623 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:28.009847 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:28.009852 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:28.009977 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.010062 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.010197 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":109,"threads_started":1}
I20260806 22:41:28.011018 13090 heartbeater.cc:347] Connected to a master server at 127.16.235.126:33693
I20260806 22:41:28.011142 13090 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:28.011339 13090 heartbeater.cc:511] Master 127.16.235.126:33693 requested a full tablet report, sending...
I20260806 22:41:28.011845 12464 ts_manager.cc:194] Registered new tserver with Master: d8a7a4b96273441b8d5a50b97ddc52b7 (127.16.235.67:46583)
I20260806 22:41:28.012307 12464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.235.67@KRBTEST.COM'} at 127.16.235.67:46305
I20260806 22:41:28.012841 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:41:28.020583 12514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.014235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.16.235.126:33693)
0806 22:41:28.014304 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.014306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.014414 (+ 108us) server_negotiation.cc:407] Connection header received
0806 22:41:28.014484 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.014485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.014496 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.014555 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:28.015198 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.015684 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.016423 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.016618 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.016703 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.017984 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:28.017985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:28.017985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:28.017993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:28.019336 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:28.019693 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:28.019698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:28.019699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:28.019749 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:28.019973 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:28.019974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:28.019974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:28.020125 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:28.020229 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.020389 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.020474 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:41:28.022115 12464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:41:28.023949 12464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:41:28.030925 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.026488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:46691 (local address 127.0.0.1:47134)
0806 22:41:28.026755 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.026804 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.026883 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.027441 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.027442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.027445 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:28.027485 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.027486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.028732 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.028733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.029805 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.029808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.030555 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.030556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.030643 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.030696 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.030765 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.030821 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":101,"threads_started":1}
I20260806 22:41:28.031576 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.026738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.16.235.65:46691)
0806 22:41:28.027151 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.027156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.027166 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:41:28.027339 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.027344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.027364 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.027441 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:28.027552 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.028603 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.029900 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.030445 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.030539 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.030625 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.031195 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.031241 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":106,"threads_started":1}
I20260806 22:41:28.031796 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.026603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.16.235.66:41705)
0806 22:41:28.026856 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.026861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.027032 (+ 171us) server_negotiation.cc:407] Connection header received
0806 22:41:28.027092 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.027094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.027111 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.027193 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:28.027416 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.028611 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.030720 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.031366 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.031456 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.031514 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.031641 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.031707 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":130,"threads_started":1}
I20260806 22:41:28.031893 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.026970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.16.235.67:46583)
0806 22:41:28.027343 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.027347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.027358 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:41:28.027397 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.027400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.027416 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.027492 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:28.027683 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.028370 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.030686 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.031416 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.031489 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.031549 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.031762 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.031807 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":95,"threads_started":1}
I20260806 22:41:28.031893 13108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.026739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:46583 (local address 127.0.0.1:39450)
0806 22:41:28.026963 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.026972 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.027020 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.027569 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.027570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.027572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:28.027602 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.027603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.028486 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.028487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.029805 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.029808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.031587 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.031588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.031654 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.031679 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.031743 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.031786 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":79,"threads_started":1}
I20260806 22:41:28.032483 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.026488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:41705 (local address 127.0.0.1:58830)
0806 22:41:28.026952 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.026963 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.027027 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.027209 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.027213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.027215 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:28.027297 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.027300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.029912 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.029912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.030587 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.030588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.031469 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.031470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.031537 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.031541 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.032350 (+ 809us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.032403 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260806 22:41:28.032553 12645 tablet_service.cc:1467] Processing CreateTablet for tablet 1d61f9d49ec446ada4173d9ae49e117a (DEFAULT_TABLE table=test-table [id=7d258cc8ac3a467f966858bd4289dfbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:28.032625 13007 tablet_service.cc:1467] Processing CreateTablet for tablet 1d61f9d49ec446ada4173d9ae49e117a (DEFAULT_TABLE table=test-table [id=7d258cc8ac3a467f966858bd4289dfbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:28.032994 12645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d61f9d49ec446ada4173d9ae49e117a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:28.033007 13007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d61f9d49ec446ada4173d9ae49e117a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:28.033386 12823 tablet_service.cc:1467] Processing CreateTablet for tablet 1d61f9d49ec446ada4173d9ae49e117a (DEFAULT_TABLE table=test-table [id=7d258cc8ac3a467f966858bd4289dfbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:28.033692 12823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d61f9d49ec446ada4173d9ae49e117a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:28.035908 13118 tablet_bootstrap.cc:492] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: Bootstrap starting.
I20260806 22:41:28.036520 13121 tablet_bootstrap.cc:492] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40: Bootstrap starting.
I20260806 22:41:28.036705 13118 tablet_bootstrap.cc:654] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:28.037137 13122 tablet_bootstrap.cc:492] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: Bootstrap starting.
I20260806 22:41:28.037345 13121 tablet_bootstrap.cc:654] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:28.037348 13118 log.cc:824] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:28.037904 13122 tablet_bootstrap.cc:654] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:28.037953 13121 log.cc:824] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:28.038200 13118 tablet_bootstrap.cc:492] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: No bootstrap required, opened a new log
I20260806 22:41:28.038360 13118 ts_tablet_manager.cc:1397] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:41:28.038582 13122 log.cc:824] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:28.038829 13121 tablet_bootstrap.cc:492] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40: No bootstrap required, opened a new log
I20260806 22:41:28.038928 13121 ts_tablet_manager.cc:1397] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:41:28.039459 13122 tablet_bootstrap.cc:492] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: No bootstrap required, opened a new log
I20260806 22:41:28.039579 13122 ts_tablet_manager.cc:1397] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:41:28.040320 13121 raft_consensus.cc:348] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } }
I20260806 22:41:28.040506 13121 raft_consensus.cc:374] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:28.040577 13121 raft_consensus.cc:729] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd9d487cba8c403d90fb23cd20766a40, State: Initialized, Role: FOLLOWER
I20260806 22:41:28.040535 13118 raft_consensus.cc:348] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } }
I20260806 22:41:28.040715 13121 consensus_queue.cc:260] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } }
I20260806 22:41:28.040795 13118 raft_consensus.cc:374] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:28.040850 13118 raft_consensus.cc:729] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a7a4b96273441b8d5a50b97ddc52b7, State: Initialized, Role: FOLLOWER
I20260806 22:41:28.040988 13118 consensus_queue.cc:260] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } }
I20260806 22:41:28.041049 13121 ts_tablet_manager.cc:1428] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:41:28.041199 12731 heartbeater.cc:503] Master 127.16.235.126:33693 was elected leader, sending a full tablet report...
I20260806 22:41:28.041296 13118 ts_tablet_manager.cc:1428] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:41:28.041440 13090 heartbeater.cc:503] Master 127.16.235.126:33693 was elected leader, sending a full tablet report...
I20260806 22:41:28.041726 13122 raft_consensus.cc:348] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } }
I20260806 22:41:28.041917 13122 raft_consensus.cc:374] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:28.041956 13122 raft_consensus.cc:729] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55e64996f954b8990fbb5d4413b0b8e, State: Initialized, Role: FOLLOWER
I20260806 22:41:28.042127 13122 consensus_queue.cc:260] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } }
I20260806 22:41:28.042467 12914 heartbeater.cc:503] Master 127.16.235.126:33693 was elected leader, sending a full tablet report...
I20260806 22:41:28.042448 13122 ts_tablet_manager.cc:1428] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 22:41:28.065879 12915 tablet.cc:2369] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:28.078507 13129 raft_consensus.cc:482] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:28.078712 13129 raft_consensus.cc:504] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } }
I20260806 22:41:28.079136 13129 leader_election.cc:288] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd9d487cba8c403d90fb23cd20766a40 (127.16.235.65:46691), d8a7a4b96273441b8d5a50b97ddc52b7 (127.16.235.67:46583)
I20260806 22:41:28.082223 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.079521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:54285 (local address 127.16.235.65:46691)
0806 22:41:28.079605 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.079608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.079622 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:41:28.079747 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.079748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.079760 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.079842 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:28.080015 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.080501 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.081468 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.081855 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.081946 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.081956 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.082110 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.082148 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:41:28.082223 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.079443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:46691 (local address 127.16.235.66:54285)
0806 22:41:28.079519 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.079536 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.079623 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.079850 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.079851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.079853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:28.079935 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.079937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.080599 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.080601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.081365 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.081367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.081944 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.081945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.082030 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.082033 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.082104 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.082148 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 22:41:28.082360 13131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.079586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:46583 (local address 127.16.235.66:34151)
0806 22:41:28.079851 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.079861 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.079941 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.080075 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.080076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.080077 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:28.080153 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.080155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.080789 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.080790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.081521 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.081523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.082085 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.082087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.082162 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.082166 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.082235 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.082274 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":190,"threads_started":1}
I20260806 22:41:28.082350 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.079694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:34151 (local address 127.16.235.67:46583)
0806 22:41:28.079768 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.079770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.079931 (+ 161us) server_negotiation.cc:407] Connection header received
0806 22:41:28.080012 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.080013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.080022 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.080063 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:28.080229 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.080700 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.081617 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.081996 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.082085 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.082093 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.082253 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.082287 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:41:28.082628 12670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d61f9d49ec446ada4173d9ae49e117a" candidate_uuid: "a55e64996f954b8990fbb5d4413b0b8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd9d487cba8c403d90fb23cd20766a40" is_pre_election: true
I20260806 22:41:28.082753 13037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d61f9d49ec446ada4173d9ae49e117a" candidate_uuid: "a55e64996f954b8990fbb5d4413b0b8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" is_pre_election: true
I20260806 22:41:28.082863 12670 raft_consensus.cc:2454] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a55e64996f954b8990fbb5d4413b0b8e in term 0.
I20260806 22:41:28.082942 13037 raft_consensus.cc:2454] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a55e64996f954b8990fbb5d4413b0b8e in term 0.
I20260806 22:41:28.083153 12764 leader_election.cc:302] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a55e64996f954b8990fbb5d4413b0b8e, fd9d487cba8c403d90fb23cd20766a40; no voters:
I20260806 22:41:28.083336 13129 raft_consensus.cc:2781] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:41:28.083412 13129 raft_consensus.cc:482] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:41:28.083431 13129 raft_consensus.cc:3037] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:28.084214 13129 raft_consensus.cc:504] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } }
I20260806 22:41:28.084378 13129 leader_election.cc:288] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [CANDIDATE]: Term 1 election: Requested vote from peers fd9d487cba8c403d90fb23cd20766a40 (127.16.235.65:46691), d8a7a4b96273441b8d5a50b97ddc52b7 (127.16.235.67:46583)
I20260806 22:41:28.084671 12670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d61f9d49ec446ada4173d9ae49e117a" candidate_uuid: "a55e64996f954b8990fbb5d4413b0b8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd9d487cba8c403d90fb23cd20766a40"
I20260806 22:41:28.084717 13037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d61f9d49ec446ada4173d9ae49e117a" candidate_uuid: "a55e64996f954b8990fbb5d4413b0b8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7"
I20260806 22:41:28.084808 13037 raft_consensus.cc:3037] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:28.084805 12670 raft_consensus.cc:3037] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:28.085572 13037 raft_consensus.cc:2454] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a55e64996f954b8990fbb5d4413b0b8e in term 1.
I20260806 22:41:28.085606 12670 raft_consensus.cc:2454] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a55e64996f954b8990fbb5d4413b0b8e in term 1.
I20260806 22:41:28.085840 12764 leader_election.cc:302] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a55e64996f954b8990fbb5d4413b0b8e, fd9d487cba8c403d90fb23cd20766a40; no voters:
I20260806 22:41:28.085939 13129 raft_consensus.cc:2781] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:28.086122 13129 raft_consensus.cc:686] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 1 LEADER]: Becoming Leader. State: Replica: a55e64996f954b8990fbb5d4413b0b8e, State: Running, Role: LEADER
I20260806 22:41:28.086238 13129 consensus_queue.cc:237] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } }
I20260806 22:41:28.087147 12464 catalog_manager.cc:5495] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e reported cstate change: term changed from 0 to 1, leader changed from <none> to a55e64996f954b8990fbb5d4413b0b8e (127.16.235.66). New cstate: current_term: 1 leader_uuid: "a55e64996f954b8990fbb5d4413b0b8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a55e64996f954b8990fbb5d4413b0b8e" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 41705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 } health_report { overall_health: UNKNOWN } } }
I20260806 22:41:28.105423 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.102595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.16.235.66:41705)
0806 22:41:28.102744 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.102746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.102758 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:41:28.102808 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.102809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.102834 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.102947 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:41:28.103137 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.103922 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.104711 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.104913 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.104980 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.105111 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:41:28.105217 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:41:28.105282 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.105321 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.105341 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 22:41:28.107546 12670 raft_consensus.cc:1261] T 1d61f9d49ec446ada4173d9ae49e117a P fd9d487cba8c403d90fb23cd20766a40 [term 1 FOLLOWER]: Refusing update from remote peer a55e64996f954b8990fbb5d4413b0b8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:28.107546 13037 raft_consensus.cc:1261] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [term 1 FOLLOWER]: Refusing update from remote peer a55e64996f954b8990fbb5d4413b0b8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:28.107976 13132 consensus_queue.cc:1035] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd9d487cba8c403d90fb23cd20766a40" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 46691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:28.108134 13129 consensus_queue.cc:1035] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8a7a4b96273441b8d5a50b97ddc52b7" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 46583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:28.111629 13139 mvcc.cc:205] Tried to move back new op lower bound from 7315685736885145600 to 7315685736801841152. Current Snapshot: MvccSnapshot[applied={T|T < 7315685736885145600}]
I20260806 22:41:28.113355 12463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:28.113462 12463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:28.114233 12463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac6a01a9543d4061818a6b5a2d039cc6: Sending DeleteTablet for 3 replicas of tablet 1d61f9d49ec446ada4173d9ae49e117a
I20260806 22:41:28.114602 12823 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d61f9d49ec446ada4173d9ae49e117a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:28 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:58830
I20260806 22:41:28.114619 13007 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d61f9d49ec446ada4173d9ae49e117a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:28 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:39450
I20260806 22:41:28.114758 12645 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d61f9d49ec446ada4173d9ae49e117a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:28 UTC) from {username='kudu', principal='kudu/127.16.235.126@KRBTEST.COM'} at 127.0.0.1:47134
I20260806 22:41:28.114889 13148 tablet_replica.cc:330] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: stopping tablet replica
I20260806 22:41:28.114993 13148 raft_consensus.cc:2229] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:41:28.115090 13148 raft_consensus.cc:2258] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:28.115231 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 12547
I20260806 22:41:28.115254 13149 tablet_replica.cc:330] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: stopping tablet replica
I20260806 22:41:28.115958 13149 raft_consensus.cc:2229] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:41:28.116086 13149 raft_consensus.cc:2258] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:28.120635 13149 ts_tablet_manager.cc:1905] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:28.128526 13148 ts_tablet_manager.cc:1905] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:28.129938 13149 ts_tablet_manager.cc:1918] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:28.130065 13149 log.cc:1196] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/1d61f9d49ec446ada4173d9ae49e117a
I20260806 22:41:28.130282 13148 ts_tablet_manager.cc:1918] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:28.130398 13149 ts_tablet_manager.cc:1939] T 1d61f9d49ec446ada4173d9ae49e117a P d8a7a4b96273441b8d5a50b97ddc52b7: Deleting consensus metadata
I20260806 22:41:28.130457 13148 log.cc:1196] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/1d61f9d49ec446ada4173d9ae49e117a
I20260806 22:41:28.130851 13148 ts_tablet_manager.cc:1939] T 1d61f9d49ec446ada4173d9ae49e117a P a55e64996f954b8990fbb5d4413b0b8e: Deleting consensus metadata
I20260806 22:41:28.131060 12447 catalog_manager.cc:4841] TS d8a7a4b96273441b8d5a50b97ddc52b7 (127.16.235.67:46583): tablet 1d61f9d49ec446ada4173d9ae49e117a (table test-table [id=7d258cc8ac3a467f966858bd4289dfbc]) successfully deleted
I20260806 22:41:28.131309 12446 catalog_manager.cc:4841] TS a55e64996f954b8990fbb5d4413b0b8e (127.16.235.66:41705): tablet 1d61f9d49ec446ada4173d9ae49e117a (table test-table [id=7d258cc8ac3a467f966858bd4289dfbc]) successfully deleted
W20260806 22:41:28.133431 12447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:46691 (error 108)
I20260806 22:41:28.133641 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 12738
I20260806 22:41:28.137130 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.136986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:46691 (local address 127.0.0.1:53838)
0806 22:41:28.137042 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.137085 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:41:28.137351 12447 catalog_manager.cc:4584] TS fd9d487cba8c403d90fb23cd20766a40 (127.16.235.65:46691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d61f9d49ec446ada4173d9ae49e117a: Network error: Client connection negotiation failed: client connection to 127.16.235.65:46691: connect: Connection refused (error 111)
I20260806 22:41:28.152768 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 12919
I20260806 22:41:28.162525 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 12417
2026-08-06T22:41:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2220 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T22:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:28Z Disabled control of system clock
I20260806 22:41:28.195865 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34141
--webserver_interface=127.16.235.126
--webserver_port=0
--builtin_ntp_servers=127.16.235.84:34035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:41:28.281574 13159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:28.281843 13159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:28.281867 13159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:28.283061 13159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:28.283102 13159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:28.283114 13159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:28.283128 13159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:28.283140 13159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:28.284494 13159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:34035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:34141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:28.284721 13159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:28.284991 13159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:28.290254 13167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:28.290232 13171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:28.290232 13168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:28.290666 13159 server_base.cc:1034] running on GCE node
I20260806 22:41:28.290958 13159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:28.291280 13159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:28.292456 13159 hybrid_clock.cc:648] HybridClock initialized: now 1786056088292446 us; error 31 us; skew 500 ppm
I20260806 22:41:28.293778 13159 webserver.cc:460] Webserver started at http://127.16.235.126:42555/ using document root <none> and password file <none>
I20260806 22:41:28.294056 13159 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:28.294127 13159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:28.294283 13159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:28.295528 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "49f75ecaef364dd1bfb7b582acb967d8"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "4de128f6aef0ffc65900946bd260b2bf"
server_key_iv: "d9ad729deeae7b47c29e8ecafdfca1a8"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.295920 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "49f75ecaef364dd1bfb7b582acb967d8"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "4de128f6aef0ffc65900946bd260b2bf"
server_key_iv: "d9ad729deeae7b47c29e8ecafdfca1a8"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.297340 13159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:41:28.298125 13181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.298326 13159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:28.298439 13159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "49f75ecaef364dd1bfb7b582acb967d8"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "4de128f6aef0ffc65900946bd260b2bf"
server_key_iv: "d9ad729deeae7b47c29e8ecafdfca1a8"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.298518 13159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:28.322839 13159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:28.323226 13159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:28.323408 13159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:28.327994 13159 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:34141
I20260806 22:41:28.328039 13243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:34141 every 8 connection(s)
I20260806 22:41:28.328423 13159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:28.329531 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:28.331274 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 13159
I20260806 22:41:28.331369 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:28.331589 17325 external_mini_cluster.cc:1428] Setting key 67cb02dc84dad5ec732abe41f84a9895
I20260806 22:41:28.332450 13244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8: Bootstrap starting.
I20260806 22:41:28.333199 13244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:28.333559 13244 log.cc:824] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:28.334421 13244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8: No bootstrap required, opened a new log
I20260806 22:41:28.336730 13244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f75ecaef364dd1bfb7b582acb967d8" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34141 } }
I20260806 22:41:28.336926 13244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:28.336969 13244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f75ecaef364dd1bfb7b582acb967d8, State: Initialized, Role: FOLLOWER
I20260806 22:41:28.337101 13244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f75ecaef364dd1bfb7b582acb967d8" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34141 } }
I20260806 22:41:28.337195 13244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:28.337248 13244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:28.337306 13244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:28.338156 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.332249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.16.235.126:34141)
0806 22:41:28.332490 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.332499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.332536 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:41:28.333161 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.333172 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.333276 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.333448 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:28.333691 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.335077 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.335858 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.336550 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.336686 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.337394 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:28.337398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:28.337399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:28.337622 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:28.337664 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:28.337678 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:28.337756 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.337823 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.337910 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260806 22:41:28.338128 13244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f75ecaef364dd1bfb7b582acb967d8" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34141 } }
I20260806 22:41:28.338274 13244 leader_election.cc:302] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49f75ecaef364dd1bfb7b582acb967d8; no voters:
I20260806 22:41:28.338460 13244 leader_election.cc:288] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:28.338541 13251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:28.338738 13251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [term 1 LEADER]: Becoming Leader. State: Replica: 49f75ecaef364dd1bfb7b582acb967d8, State: Running, Role: LEADER
I20260806 22:41:28.338811 13244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:28.338883 13251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f75ecaef364dd1bfb7b582acb967d8" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34141 } }
I20260806 22:41:28.339249 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49f75ecaef364dd1bfb7b582acb967d8. Latest consensus state: current_term: 1 leader_uuid: "49f75ecaef364dd1bfb7b582acb967d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f75ecaef364dd1bfb7b582acb967d8" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34141 } } }
I20260806 22:41:28.339295 13252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49f75ecaef364dd1bfb7b582acb967d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49f75ecaef364dd1bfb7b582acb967d8" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34141 } } }
I20260806 22:41:28.339461 13252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:28.339821 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:28.339957 13264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:28.340447 13264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:28.342067 13264 catalog_manager.cc:1347] Generated new cluster ID: 6a16156f85e343aca60d585bfd9db900
I20260806 22:41:28.342131 13264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:28.355062 13264 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:28.355692 13264 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:28.362143 13264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8: Generated new TSK 0
I20260806 22:41:28.362388 13264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:28.368278 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.65:0
--local_ip_for_outbound_sockets=127.16.235.65
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34141
--builtin_ntp_servers=127.16.235.84:34035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:41:28.467286 13277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:28.467537 13277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:28.467563 13277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:28.468854 13277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:28.468919 13277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:28.468948 13277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.65
I20260806 22:41:28.470458 13277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:34035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.235.65
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.65
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:28.470793 13277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:28.471186 13277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:28.476735 13282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:28.477017 13283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:28.476742 13287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:28.477306 13277 server_base.cc:1034] running on GCE node
I20260806 22:41:28.477561 13277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:28.477861 13277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:28.479108 13277 hybrid_clock.cc:648] HybridClock initialized: now 1786056088479092 us; error 37 us; skew 500 ppm
I20260806 22:41:28.480710 13277 webserver.cc:460] Webserver started at http://127.16.235.65:38247/ using document root <none> and password file <none>
I20260806 22:41:28.481030 13277 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:28.481082 13277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:28.481266 13277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:28.482450 13277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/data/instance:
uuid: "d43dc6d27c904fbba73bae7438a9af90"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "1eb02d4fafa0c23d3f86f269aca97adc"
server_key_iv: "1bd24f94d9390ec758cbd2563cb1e5c6"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.482887 13277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance:
uuid: "d43dc6d27c904fbba73bae7438a9af90"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "1eb02d4fafa0c23d3f86f269aca97adc"
server_key_iv: "1bd24f94d9390ec758cbd2563cb1e5c6"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.484346 13277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:28.485203 13295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.485379 13277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:28.485502 13277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/wal
uuid: "d43dc6d27c904fbba73bae7438a9af90"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "1eb02d4fafa0c23d3f86f269aca97adc"
server_key_iv: "1bd24f94d9390ec758cbd2563cb1e5c6"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.485586 13277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:28.506150 13277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:28.506533 13277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:28.506736 13277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:28.507041 13277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:28.507421 13277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:28.507478 13277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.507524 13277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:28.507552 13277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.513753 13277 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.65:35377
I20260806 22:41:28.513801 13449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.65:35377 every 8 connection(s)
I20260806 22:41:28.514173 13277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/data/info.pb
I20260806 22:41:28.518819 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 13277
I20260806 22:41:28.518946 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-0/wal/instance
I20260806 22:41:28.519183 17325 external_mini_cluster.cc:1428] Setting key 349a0765858ae81715acd843868350f6
I20260806 22:41:28.519802 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.515054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.65:56861 (local address 127.16.235.126:34141)
0806 22:41:28.515134 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.515136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.515391 (+ 255us) server_negotiation.cc:407] Connection header received
0806 22:41:28.515926 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.515927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.515944 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.516008 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:28.516699 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.517213 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.518183 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.518366 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.518418 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.519487 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:28.519489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:28.519489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:28.519494 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:28.519510 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:28.519514 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:28.519591 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.519690 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.519721 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:41:28.520072 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.515042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34141 (local address 127.16.235.65:56861)
0806 22:41:28.515242 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.515281 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.515817 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.516076 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.516080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.516088 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:28.516619 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.516623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.517301 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.517302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.518098 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.518100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.518447 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.518448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.518613 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.519305 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:41:28.519306 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:28.519347 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:41:28.519378 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:28.519592 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:28.519600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:28.519604 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.519692 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.519807 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260806 22:41:28.520371 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.66:0
--local_ip_for_outbound_sockets=127.16.235.66
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34141
--builtin_ntp_servers=127.16.235.84:34035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:41:28.520670 13450 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34141
I20260806 22:41:28.520802 13450 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:28.521006 13450 heartbeater.cc:511] Master 127.16.235.126:34141 requested a full tablet report, sending...
I20260806 22:41:28.521579 13204 ts_manager.cc:194] Registered new tserver with Master: d43dc6d27c904fbba73bae7438a9af90 (127.16.235.65:35377)
I20260806 22:41:28.522233 13204 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.65:56861
W20260806 22:41:28.609052 13454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:28.609292 13454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:28.609318 13454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:28.610522 13454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:28.610570 13454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:28.610597 13454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.66
I20260806 22:41:28.612052 13454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:34035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.235.66
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.66
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:28.612334 13454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:28.612666 13454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:28.619161 13464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:28.619169 13460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:28.619172 13461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:28.619450 13454 server_base.cc:1034] running on GCE node
I20260806 22:41:28.619727 13454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:28.620007 13454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:28.621191 13454 hybrid_clock.cc:648] HybridClock initialized: now 1786056088621176 us; error 38 us; skew 500 ppm
I20260806 22:41:28.622500 13454 webserver.cc:460] Webserver started at http://127.16.235.66:32967/ using document root <none> and password file <none>
I20260806 22:41:28.622826 13454 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:28.622900 13454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:28.623054 13454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:28.624225 13454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/data/instance:
uuid: "36d3607ad1744973a774095e7d779b89"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "979c2861a80e2306189d5d7a4c629707"
server_key_iv: "572a21c9443e2fe9e004eb082990708c"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.624665 13454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance:
uuid: "36d3607ad1744973a774095e7d779b89"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "979c2861a80e2306189d5d7a4c629707"
server_key_iv: "572a21c9443e2fe9e004eb082990708c"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.626086 13454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:28.627065 13472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.627282 13454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:28.627368 13454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/wal
uuid: "36d3607ad1744973a774095e7d779b89"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "979c2861a80e2306189d5d7a4c629707"
server_key_iv: "572a21c9443e2fe9e004eb082990708c"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.627432 13454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:28.642376 13454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:28.642848 13454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:28.642997 13454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:28.643253 13454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:28.643680 13454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:28.643726 13454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.643752 13454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:28.643810 13454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.650519 13454 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.66:35685
I20260806 22:41:28.650576 13627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.66:35685 every 8 connection(s)
I20260806 22:41:28.651014 13454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/data/info.pb
I20260806 22:41:28.656913 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.651896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.66:44817 (local address 127.16.235.126:34141)
0806 22:41:28.651994 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.651998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.652232 (+ 234us) server_negotiation.cc:407] Connection header received
0806 22:41:28.652803 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.652804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.652815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.652868 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:28.653649 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.654179 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.655315 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.655505 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.655567 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.656484 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:28.656486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:28.656486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:28.656491 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:28.656506 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:28.656510 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:28.656559 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.656726 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.656751 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:41:28.657024 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.651835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34141 (local address 127.16.235.66:44817)
0806 22:41:28.652073 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.652116 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.652691 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.652937 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.652941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.652950 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:28.653536 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.653541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.654290 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.654291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.655211 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.655214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.655595 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.655596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.655731 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.656317 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:41:28.656317 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:28.656350 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:41:28.656378 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:28.656601 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:28.656609 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:28.656612 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.656696 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.656793 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260806 22:41:28.657051 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 13454
I20260806 22:41:28.657168 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/wal/instance
I20260806 22:41:28.657421 17325 external_mini_cluster.cc:1428] Setting key bdb6024b8224092c32b777506648bd2d
I20260806 22:41:28.657559 13628 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34141
I20260806 22:41:28.657675 13628 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:28.657881 13628 heartbeater.cc:511] Master 127.16.235.126:34141 requested a full tablet report, sending...
I20260806 22:41:28.658318 13204 ts_manager.cc:194] Registered new tserver with Master: 36d3607ad1744973a774095e7d779b89 (127.16.235.66:35685)
I20260806 22:41:28.658615 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.235.67:0
--local_ip_for_outbound_sockets=127.16.235.67
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34141
--builtin_ntp_servers=127.16.235.84:34035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:41:28.658855 13204 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.66:44817
W20260806 22:41:28.745005 13633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:28.745240 13633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:28.745263 13633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:28.746357 13633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:28.746390 13633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:28.746415 13633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.235.67
I20260806 22:41:28.747737 13633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:34035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.235.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.235.67
--webserver_port=0
--tserver_master_addrs=127.16.235.126:34141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.235.67
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:28.747946 13633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:28.748217 13633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:28.753499 13644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:28.753571 13639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:28.753499 13641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:28.753669 13633 server_base.cc:1034] running on GCE node
I20260806 22:41:28.754029 13633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:28.754319 13633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:28.755496 13633 hybrid_clock.cc:648] HybridClock initialized: now 1786056088755483 us; error 48 us; skew 500 ppm
I20260806 22:41:28.756871 13633 webserver.cc:460] Webserver started at http://127.16.235.67:44429/ using document root <none> and password file <none>
I20260806 22:41:28.757146 13633 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:28.757223 13633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:28.757392 13633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:28.758546 13633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/data/instance:
uuid: "24e4849b8b604b24b74858190efe4029"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "0b1369ddc63f007ae33beb23df52579b"
server_key_iv: "cca8037d2181254d6a795e3783e71e85"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.759017 13633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance:
uuid: "24e4849b8b604b24b74858190efe4029"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "0b1369ddc63f007ae33beb23df52579b"
server_key_iv: "cca8037d2181254d6a795e3783e71e85"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.760455 13633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:28.761198 13652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.761377 13633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:41:28.761492 13633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/wal
uuid: "24e4849b8b604b24b74858190efe4029"
format_stamp: "Formatted at 2026-08-06 22:41:28 on dist-test-slave-3kk6"
server_key: "0b1369ddc63f007ae33beb23df52579b"
server_key_iv: "cca8037d2181254d6a795e3783e71e85"
server_key_version: "encryptionkey@0"
I20260806 22:41:28.761572 13633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:28.776501 13633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:28.776901 13633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:28.777077 13633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:28.777354 13633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:41:28.777732 13633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:41:28.777792 13633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.777840 13633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:41:28.777864 13633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:28.784293 13633 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.67:42257
I20260806 22:41:28.784343 13819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.67:42257 every 8 connection(s)
I20260806 22:41:28.784780 13633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/data/info.pb
I20260806 22:41:28.790217 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.785600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:51533 (local address 127.16.235.126:34141)
0806 22:41:28.785678 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.785680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.785912 (+ 232us) server_negotiation.cc:407] Connection header received
0806 22:41:28.786405 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.786406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.786417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.786475 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:28.787280 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.787789 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.788849 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.789034 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.789091 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.789979 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:28.789980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:28.789980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:28.789984 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:28.789999 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:28.790002 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:28.790046 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.790131 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.790153 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:41:28.790447 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.785589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:34141 (local address 127.16.235.67:51533)
0806 22:41:28.785783 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.785819 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.786313 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.786560 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.786564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.786573 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:28.787193 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.787199 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.787883 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.787883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.788725 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.788728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.789116 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.789117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.789283 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.789844 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:41:28.789844 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:28.789872 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:41:28.789897 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:28.790059 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:28.790063 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:28.790066 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.790118 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.790200 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260806 22:41:28.790992 13820 heartbeater.cc:347] Connected to a master server at 127.16.235.126:34141
I20260806 22:41:28.791091 13820 heartbeater.cc:464] Registering TS with master...
I20260806 22:41:28.791249 13820 heartbeater.cc:511] Master 127.16.235.126:34141 requested a full tablet report, sending...
I20260806 22:41:28.791674 13204 ts_manager.cc:194] Registered new tserver with Master: 24e4849b8b604b24b74858190efe4029 (127.16.235.67:42257)
I20260806 22:41:28.792083 13204 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.235.67:51533
I20260806 22:41:28.794992 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 13633
I20260806 22:41:28.795138 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/wal/instance
I20260806 22:41:28.795353 17325 external_mini_cluster.cc:1428] Setting key 213943f7ec152a50c911c109f5787db1
I20260806 22:41:28.796386 17325 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:41:28.801090 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.797932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.16.235.126:34141)
0806 22:41:28.798038 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.798040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.798100 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:41:28.798183 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.798184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.798195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.798263 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:28.798432 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.799012 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.799803 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.800001 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.800080 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.800812 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:28.800813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:28.800814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:28.800819 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:28.800841 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:28.800845 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:28.800905 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.800971 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.800994 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:41:28.802948 13204 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:41:28.804622 13204 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:41:28.811424 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.806987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.67:42257 (local address 127.0.0.1:47762)
0806 22:41:28.807550 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.807560 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.807614 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.807899 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.807902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.807904 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:28.807978 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.807980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.808902 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.808904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.810144 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.810147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.811104 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.811105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.811188 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.811202 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.811279 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.811345 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260806 22:41:28.811434 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.807108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.16.235.67:42257)
0806 22:41:28.807555 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.807559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.807602 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:41:28.807688 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.807691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.807708 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.807805 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:28.808062 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.808796 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.810253 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.811002 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.811091 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.811193 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.811296 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.811336 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":106,"threads_started":1}
I20260806 22:41:28.811823 13835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.807239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35685 (local address 127.0.0.1:38280)
0806 22:41:28.807647 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.807657 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.807706 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.807966 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.807966 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.807968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:28.808012 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.808014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.809334 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.809336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.810440 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.810442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.811524 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.811524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.811607 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.811610 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.811684 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.811731 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":24,"thread_start_us":110,"threads_started":1}
I20260806 22:41:28.811839 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.807109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.16.235.65:35377)
0806 22:41:28.807424 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.807437 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.807448 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:41:28.807752 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.807756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.807775 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.807833 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:28.808077 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.809232 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.810568 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.811330 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.811394 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.811481 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.811725 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.811764 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":102,"threads_started":1}
I20260806 22:41:28.812116 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.806987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:35377 (local address 127.0.0.1:51730)
0806 22:41:28.807329 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:28.807374 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:28.807452 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:28.807936 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:28.807937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:28.807939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:28.807989 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.807991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.809351 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.809352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.810444 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:28.810447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.811522 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.811523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:28.811604 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.811608 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:28.811984 (+ 376us) client_negotiation.cc:240] Negotiation successful
0806 22:41:28.812030 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":61,"thread_start_us":95,"threads_started":1}
I20260806 22:41:28.811844 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:28.807368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.16.235.66:35685)
0806 22:41:28.807583 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:28.807587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:28.807764 (+ 177us) server_negotiation.cc:407] Connection header received
0806 22:41:28.807820 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:28.807822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:28.807836 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:28.807902 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:28.808093 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.809232 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.810531 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:28.811315 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:28.811393 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:28.811481 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:28.811714 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 22:41:28.811757 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":115,"threads_started":1}
I20260806 22:41:28.812813 13359 tablet_service.cc:1467] Processing CreateTablet for tablet cf18018511fb4d8c85197a022ca1c3eb (DEFAULT_TABLE table=test-table [id=ffbeae05cd2c48e9b33ad0db8524167f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:28.812803 13716 tablet_service.cc:1467] Processing CreateTablet for tablet cf18018511fb4d8c85197a022ca1c3eb (DEFAULT_TABLE table=test-table [id=ffbeae05cd2c48e9b33ad0db8524167f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:28.812801 13536 tablet_service.cc:1467] Processing CreateTablet for tablet cf18018511fb4d8c85197a022ca1c3eb (DEFAULT_TABLE table=test-table [id=ffbeae05cd2c48e9b33ad0db8524167f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:41:28.813200 13359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf18018511fb4d8c85197a022ca1c3eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:28.813197 13536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf18018511fb4d8c85197a022ca1c3eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:28.813197 13716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf18018511fb4d8c85197a022ca1c3eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:28.816220 13837 tablet_bootstrap.cc:492] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90: Bootstrap starting.
I20260806 22:41:28.816726 13838 tablet_bootstrap.cc:492] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: Bootstrap starting.
I20260806 22:41:28.816887 13841 tablet_bootstrap.cc:492] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: Bootstrap starting.
I20260806 22:41:28.817126 13837 tablet_bootstrap.cc:654] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:28.817436 13837 log.cc:824] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:28.817456 13838 tablet_bootstrap.cc:654] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:28.817505 13841 tablet_bootstrap.cc:654] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:28.817800 13838 log.cc:824] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:28.817826 13841 log.cc:824] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:28.818264 13837 tablet_bootstrap.cc:492] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90: No bootstrap required, opened a new log
I20260806 22:41:28.818356 13837 ts_tablet_manager.cc:1397] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:41:28.818967 13838 tablet_bootstrap.cc:492] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: No bootstrap required, opened a new log
I20260806 22:41:28.819036 13838 ts_tablet_manager.cc:1397] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:41:28.819064 13841 tablet_bootstrap.cc:492] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: No bootstrap required, opened a new log
I20260806 22:41:28.819192 13841 ts_tablet_manager.cc:1397] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:41:28.820878 13838 raft_consensus.cc:348] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } }
I20260806 22:41:28.821044 13838 raft_consensus.cc:374] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:28.821092 13838 raft_consensus.cc:729] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d3607ad1744973a774095e7d779b89, State: Initialized, Role: FOLLOWER
I20260806 22:41:28.821210 13838 consensus_queue.cc:260] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } }
I20260806 22:41:28.821228 13837 raft_consensus.cc:348] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } }
I20260806 22:41:28.821412 13837 raft_consensus.cc:374] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:28.821465 13837 raft_consensus.cc:729] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d43dc6d27c904fbba73bae7438a9af90, State: Initialized, Role: FOLLOWER
I20260806 22:41:28.821522 13838 ts_tablet_manager.cc:1428] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:41:28.821645 13628 heartbeater.cc:503] Master 127.16.235.126:34141 was elected leader, sending a full tablet report...
I20260806 22:41:28.821612 13837 consensus_queue.cc:260] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } }
I20260806 22:41:28.821905 13837 ts_tablet_manager.cc:1428] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:41:28.822032 13450 heartbeater.cc:503] Master 127.16.235.126:34141 was elected leader, sending a full tablet report...
I20260806 22:41:28.821800 13841 raft_consensus.cc:348] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } }
I20260806 22:41:28.822389 13841 raft_consensus.cc:374] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:28.822449 13841 raft_consensus.cc:729] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24e4849b8b604b24b74858190efe4029, State: Initialized, Role: FOLLOWER
I20260806 22:41:28.822572 13841 consensus_queue.cc:260] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } }
I20260806 22:41:28.822880 13841 ts_tablet_manager.cc:1428] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 22:41:28.822983 13820 heartbeater.cc:503] Master 127.16.235.126:34141 was elected leader, sending a full tablet report...
W20260806 22:41:28.901654 13629 tablet.cc:2369] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:41:29.014978 13452 tablet.cc:2369] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:29.034439 13848 raft_consensus.cc:482] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:41:29.034614 13848 raft_consensus.cc:504] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } }
I20260806 22:41:29.035070 13848 leader_election.cc:288] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d43dc6d27c904fbba73bae7438a9af90 (127.16.235.65:35377), 36d3607ad1744973a774095e7d779b89 (127.16.235.66:35685)
W20260806 22:41:29.035400 13821 tablet.cc:2369] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:41:29.038021 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:29.035335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:35377 (local address 127.16.235.67:47297)
0806 22:41:29.035403 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:29.035418 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:29.035480 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:29.035665 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:29.035666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:29.035668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:29.035775 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:29.035778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.036423 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.036425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:29.037181 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:29.037183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.037740 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.037741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:29.037831 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:29.037834 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:41:29.037900 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:41:29.037946 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:41:29.038021 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:29.035427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:47297 (local address 127.16.235.65:35377)
0806 22:41:29.035492 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:29.035494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:29.035508 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:41:29.035555 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:29.035556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:29.035568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:29.035612 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:29.035849 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.036332 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.037292 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.037645 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.037730 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:29.037739 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:29.037909 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:41:29.037954 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:41:29.038460 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf18018511fb4d8c85197a022ca1c3eb" candidate_uuid: "24e4849b8b604b24b74858190efe4029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d43dc6d27c904fbba73bae7438a9af90" is_pre_election: true
I20260806 22:41:29.038635 13385 raft_consensus.cc:2454] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e4849b8b604b24b74858190efe4029 in term 0.
I20260806 22:41:29.038686 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:29.035749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.235.67:56937 (local address 127.16.235.66:35685)
0806 22:41:29.035829 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:29.035832 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:29.035848 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:41:29.035986 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:29.035987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:29.035995 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:29.036064 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:41:29.036240 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.036741 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.037767 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.038281 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.038368 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:29.038378 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:29.038551 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:41:29.038593 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:41:29.038704 13849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:29.035473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.66:35685 (local address 127.16.235.67:56937)
0806 22:41:29.035762 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:29.035795 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:29.035872 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:29.036067 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:29.036068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:29.036069 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:41:29.036159 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:29.036161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.036841 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.036842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:29.037651 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:29.037653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.038381 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.038382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:29.038465 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:29.038469 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:41:29.038545 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:41:29.038585 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":120,"threads_started":1}
I20260806 22:41:29.038939 13658 leader_election.cc:302] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e4849b8b604b24b74858190efe4029, d43dc6d27c904fbba73bae7438a9af90; no voters:
I20260806 22:41:29.039055 13563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf18018511fb4d8c85197a022ca1c3eb" candidate_uuid: "24e4849b8b604b24b74858190efe4029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d3607ad1744973a774095e7d779b89" is_pre_election: true
I20260806 22:41:29.039120 13848 raft_consensus.cc:2781] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:41:29.039202 13848 raft_consensus.cc:482] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:41:29.039223 13563 raft_consensus.cc:2454] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24e4849b8b604b24b74858190efe4029 in term 0.
I20260806 22:41:29.039239 13848 raft_consensus.cc:3037] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:29.040227 13848 raft_consensus.cc:504] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } }
I20260806 22:41:29.040393 13848 leader_election.cc:288] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [CANDIDATE]: Term 1 election: Requested vote from peers d43dc6d27c904fbba73bae7438a9af90 (127.16.235.65:35377), 36d3607ad1744973a774095e7d779b89 (127.16.235.66:35685)
I20260806 22:41:29.040642 13563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf18018511fb4d8c85197a022ca1c3eb" candidate_uuid: "24e4849b8b604b24b74858190efe4029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d3607ad1744973a774095e7d779b89"
I20260806 22:41:29.040699 13385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf18018511fb4d8c85197a022ca1c3eb" candidate_uuid: "24e4849b8b604b24b74858190efe4029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d43dc6d27c904fbba73bae7438a9af90"
I20260806 22:41:29.040789 13385 raft_consensus.cc:3037] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:29.040764 13563 raft_consensus.cc:3037] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:29.041584 13385 raft_consensus.cc:2454] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e4849b8b604b24b74858190efe4029 in term 1.
I20260806 22:41:29.041612 13563 raft_consensus.cc:2454] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24e4849b8b604b24b74858190efe4029 in term 1.
I20260806 22:41:29.041770 13658 leader_election.cc:302] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24e4849b8b604b24b74858190efe4029, 36d3607ad1744973a774095e7d779b89; no voters:
I20260806 22:41:29.041924 13848 raft_consensus.cc:2781] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:29.042131 13848 raft_consensus.cc:686] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 1 LEADER]: Becoming Leader. State: Replica: 24e4849b8b604b24b74858190efe4029, State: Running, Role: LEADER
I20260806 22:41:29.042235 13848 consensus_queue.cc:237] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } }
I20260806 22:41:29.043151 13204 catalog_manager.cc:5495] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24e4849b8b604b24b74858190efe4029 (127.16.235.67). New cstate: current_term: 1 leader_uuid: "24e4849b8b604b24b74858190efe4029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24e4849b8b604b24b74858190efe4029" member_type: VOTER last_known_addr { host: "127.16.235.67" port: 42257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 } health_report { overall_health: UNKNOWN } } }
I20260806 22:41:29.095305 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:29.092544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.16.235.67:42257)
0806 22:41:29.092611 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:29.092613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:29.092621 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:41:29.092662 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:29.092663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:29.092682 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:29.092751 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:41:29.092978 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.093449 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.094221 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:29.094452 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:29.094510 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:29.094812 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:41:29.094995 (+ 183us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:41:29.095121 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:29.095163 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:41:29.095185 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:41:29.097667 13385 raft_consensus.cc:1261] T cf18018511fb4d8c85197a022ca1c3eb P d43dc6d27c904fbba73bae7438a9af90 [term 1 FOLLOWER]: Refusing update from remote peer 24e4849b8b604b24b74858190efe4029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:29.097666 13563 raft_consensus.cc:1261] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [term 1 FOLLOWER]: Refusing update from remote peer 24e4849b8b604b24b74858190efe4029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:29.098152 13848 consensus_queue.cc:1035] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d3607ad1744973a774095e7d779b89" member_type: VOTER last_known_addr { host: "127.16.235.66" port: 35685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:29.098345 13850 consensus_queue.cc:1035] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d43dc6d27c904fbba73bae7438a9af90" member_type: VOTER last_known_addr { host: "127.16.235.65" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:29.103520 13855 mvcc.cc:205] Tried to move back new op lower bound from 7315685740939993088 to 7315685740717703168. Current Snapshot: MvccSnapshot[applied={T|T < 7315685740939993088 or (T in {7315685740939993088})}]
I20260806 22:41:29.103791 13203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:29.103919 13203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:41:29.104748 13203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49f75ecaef364dd1bfb7b582acb967d8: Sending DeleteTablet for 3 replicas of tablet cf18018511fb4d8c85197a022ca1c3eb
I20260806 22:41:29.105131 13359 tablet_service.cc:1514] Processing DeleteTablet for tablet cf18018511fb4d8c85197a022ca1c3eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:29 UTC) from {username='slave'} at 127.0.0.1:51730
I20260806 22:41:29.105154 13716 tablet_service.cc:1514] Processing DeleteTablet for tablet cf18018511fb4d8c85197a022ca1c3eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:29 UTC) from {username='slave'} at 127.0.0.1:47762
I20260806 22:41:29.105149 13536 tablet_service.cc:1514] Processing DeleteTablet for tablet cf18018511fb4d8c85197a022ca1c3eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:41:29 UTC) from {username='slave'} at 127.0.0.1:38280
I20260806 22:41:29.105406 13868 tablet_replica.cc:330] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: stopping tablet replica
I20260806 22:41:29.105516 13868 raft_consensus.cc:2229] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:41:29.105535 13866 tablet_replica.cc:330] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: stopping tablet replica
I20260806 22:41:29.105628 13866 raft_consensus.cc:2229] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:41:29.105631 13868 raft_consensus.cc:2258] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:29.105732 13866 raft_consensus.cc:2258] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:41:29.105746 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 13277
I20260806 22:41:29.106202 13866 ts_tablet_manager.cc:1905] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:29.106278 13868 ts_tablet_manager.cc:1905] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:41:29.108237 13868 ts_tablet_manager.cc:1918] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:29.108317 13868 log.cc:1196] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-1/wal/wals/cf18018511fb4d8c85197a022ca1c3eb
I20260806 22:41:29.108613 13868 ts_tablet_manager.cc:1939] T cf18018511fb4d8c85197a022ca1c3eb P 36d3607ad1744973a774095e7d779b89: Deleting consensus metadata
I20260806 22:41:29.108717 13866 ts_tablet_manager.cc:1918] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:41:29.108810 13866 log.cc:1196] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: Deleting WAL directory at /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786055836003438-17325-0/minicluster-data/ts-2/wal/wals/cf18018511fb4d8c85197a022ca1c3eb
I20260806 22:41:29.109115 13866 ts_tablet_manager.cc:1939] T cf18018511fb4d8c85197a022ca1c3eb P 24e4849b8b604b24b74858190efe4029: Deleting consensus metadata
I20260806 22:41:29.109088 13187 catalog_manager.cc:4841] TS 36d3607ad1744973a774095e7d779b89 (127.16.235.66:35685): tablet cf18018511fb4d8c85197a022ca1c3eb (table test-table [id=ffbeae05cd2c48e9b33ad0db8524167f]) successfully deleted
I20260806 22:41:29.109578 13182 catalog_manager.cc:4841] TS 24e4849b8b604b24b74858190efe4029 (127.16.235.67:42257): tablet cf18018511fb4d8c85197a022ca1c3eb (table test-table [id=ffbeae05cd2c48e9b33ad0db8524167f]) successfully deleted
W20260806 22:41:29.119647 13187 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.235.65:35377 (error 108)
I20260806 22:41:29.119930 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 13454
I20260806 22:41:29.120307 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:29.120162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.65:35377 (local address 127.0.0.1:51734)
0806 22:41:29.120216 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:29.120266 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.65:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:41:29.120424 13187 catalog_manager.cc:4584] TS d43dc6d27c904fbba73bae7438a9af90 (127.16.235.65:35377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf18018511fb4d8c85197a022ca1c3eb: Network error: Client connection negotiation failed: client connection to 127.16.235.65:35377: connect: Connection refused (error 111)
I20260806 22:41:29.140427 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 13633
I20260806 22:41:29.155406 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 13159
2026-08-06T22:41:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (994 ms)
[----------] 6 tests from EncryptionPolicyTest (10001 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:29 dist-test-slave-3kk6 krb5kdc[13872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:29 dist-test-slave-3kk6 krb5kdc[13872](info): set up 2 sockets
Aug 06 22:41:29 dist-test-slave-3kk6 krb5kdc[13872](info): commencing operation
krb5kdc: starting...
W20260806 22:41:30.399303 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:30 dist-test-slave-3kk6 krb5kdc[13872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:30.536375 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42651
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:30.625149 13888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:30.625448 13888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:30.625494 13888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:30.627214 13888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:30.627288 13888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:30.627313 13888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:30.627332 13888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:30.627349 13888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:30.629516 13888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42651
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:30.629844 13888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:30.630116 13888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:30.635483 13894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:30.635497 13896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:30.635478 13893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:30.635524 13888 server_base.cc:1034] running on GCE node
I20260806 22:41:30.636037 13888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:30.636356 13888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:30.637509 13888 hybrid_clock.cc:648] HybridClock initialized: now 1786056090637487 us; error 42 us; skew 500 ppm
Aug 06 22:41:30 dist-test-slave-3kk6 krb5kdc[13872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056090, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:30.640357 13888 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:30.640841 13888 webserver.cc:460] Webserver started at http://127.0.0.1:32999/ using document root <none> and password file <none>
I20260806 22:41:30.641110 13888 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:30.641158 13888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:30.641347 13888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:30.642534 13888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "0d09518e302643e09d039b501c83cea2"
format_stamp: "Formatted at 2026-08-06 22:41:30 on dist-test-slave-3kk6"
server_key: "f5f0509551d035bdb45c21c493e73c3b"
server_key_iv: "50be31618a1ae86a08ca7ed215eeee95"
server_key_version: "encryptionkey@0"
I20260806 22:41:30.643028 13888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "0d09518e302643e09d039b501c83cea2"
format_stamp: "Formatted at 2026-08-06 22:41:30 on dist-test-slave-3kk6"
server_key: "f5f0509551d035bdb45c21c493e73c3b"
server_key_iv: "50be31618a1ae86a08ca7ed215eeee95"
server_key_version: "encryptionkey@0"
I20260806 22:41:30.644495 13888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:41:30.645350 13906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:30.645588 13888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:30.645709 13888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "0d09518e302643e09d039b501c83cea2"
format_stamp: "Formatted at 2026-08-06 22:41:30 on dist-test-slave-3kk6"
server_key: "f5f0509551d035bdb45c21c493e73c3b"
server_key_iv: "50be31618a1ae86a08ca7ed215eeee95"
server_key_version: "encryptionkey@0"
I20260806 22:41:30.645812 13888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:30.675051 13888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:30.677251 13888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:30.677443 13888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:30.682206 13888 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42651
I20260806 22:41:30.682242 13980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42651 every 8 connection(s)
I20260806 22:41:30.682919 13888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:30.683388 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 13888
I20260806 22:41:30.683537 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:30.683755 17325 external_mini_cluster.cc:1428] Setting key dfda7abf7bfa1f979e760beeb9cd1611
I20260806 22:41:30.683799 13981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:30.687203 13981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2: Bootstrap starting.
I20260806 22:41:30.688000 13981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:30.688331 13981 log.cc:824] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:30.689141 13981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2: No bootstrap required, opened a new log
Aug 06 22:41:30 dist-test-slave-3kk6 krb5kdc[13872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:41:30.692016 13981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09518e302643e09d039b501c83cea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42651 } }
I20260806 22:41:30.692250 13981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:30.692303 13981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d09518e302643e09d039b501c83cea2, State: Initialized, Role: FOLLOWER
I20260806 22:41:30.692468 13981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09518e302643e09d039b501c83cea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42651 } }
I20260806 22:41:30.692562 13981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:30.692587 13981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:30.692655 13981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:30.693588 13981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09518e302643e09d039b501c83cea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42651 } }
I20260806 22:41:30.693718 13981 leader_election.cc:302] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d09518e302643e09d039b501c83cea2; no voters:
I20260806 22:41:30.693930 13981 leader_election.cc:288] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:30.694008 13988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:30.694226 13981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:30.694303 13988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [term 1 LEADER]: Becoming Leader. State: Replica: 0d09518e302643e09d039b501c83cea2, State: Running, Role: LEADER
I20260806 22:41:30.694454 13988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09518e302643e09d039b501c83cea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42651 } }
I20260806 22:41:30.694990 13989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d09518e302643e09d039b501c83cea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09518e302643e09d039b501c83cea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42651 } } }
I20260806 22:41:30.695076 13989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:30.695312 13990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d09518e302643e09d039b501c83cea2. Latest consensus state: current_term: 1 leader_uuid: "0d09518e302643e09d039b501c83cea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d09518e302643e09d039b501c83cea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42651 } } }
I20260806 22:41:30.695717 13990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:30.696014 13999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:30.696719 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:30.684400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50083 (local address 127.0.0.1:42651)
0806 22:41:30.684593 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:30.684602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:30.684629 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:41:30.685308 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:30.685316 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:30.685369 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:30.685495 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:30.686388 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:30.687459 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:30.688572 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:30.689120 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:30.689260 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:30.692159 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:30.692164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:30.692165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:30.692180 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:30.694404 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:30.695065 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:30.695070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:30.695071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:30.695218 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:30.695524 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:30.695525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:30.695526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:30.695902 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:30.696210 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:30.696384 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 22:41:30.696471 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260806 22:41:30.696760 13999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:30.697814 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:30.703655 13999 catalog_manager.cc:1347] Generated new cluster ID: 1fcf8a9a3a8e441386d1fab7afbab046
I20260806 22:41:30.703745 13999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:30.710418 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:30.703123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48633 (local address 127.0.0.1:42651)
0806 22:41:30.703174 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:30.703177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:30.703747 (+ 570us) server_negotiation.cc:407] Connection header received
0806 22:41:30.703887 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:30.703889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:30.703901 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:30.703931 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:30.704568 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:30.705081 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:30.705827 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:30.706067 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:30.706155 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:30.707557 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:30.707559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:30.707559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:30.707568 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:30.709131 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:30.709539 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:30.709540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:30.709541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:30.709607 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:30.709852 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:30.709853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:30.709853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:30.709976 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:30.710093 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:30.710226 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:41:30.710304 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:41:30.713393 13999 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:30.714148 13999 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:30.721510 13999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d09518e302643e09d039b501c83cea2: Generated new TSK 0
I20260806 22:41:30.721838 13999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:30.727489 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 13888
W20260806 22:41:30.746371 13870 connection.cc:535] client connection to 127.0.0.1:42651 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42651): Connection reset by peer (error 104)
2026-08-06T22:41:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1578 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:30 dist-test-slave-3kk6 krb5kdc[14021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:30 dist-test-slave-3kk6 krb5kdc[14021](info): set up 2 sockets
Aug 06 22:41:30 dist-test-slave-3kk6 krb5kdc[14021](info): commencing operation
krb5kdc: starting...
W20260806 22:41:31.984972 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:32 dist-test-slave-3kk6 krb5kdc[14021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:32.118212 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:32.203765 14037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:32.203999 14037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:32.204022 14037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:32.205101 14037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:32.205137 14037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:32.205150 14037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:32.205163 14037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:32.205175 14037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:32.206617 14037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43531
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:32.206914 14037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:32.207113 14037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:32.212090 14045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:32.212112 14044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:32.212325 14037 server_base.cc:1034] running on GCE node
W20260806 22:41:32.212131 14047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:32.212620 14037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:32.212918 14037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:32.214090 14037 hybrid_clock.cc:648] HybridClock initialized: now 1786056092214069 us; error 39 us; skew 500 ppm
Aug 06 22:41:32 dist-test-slave-3kk6 krb5kdc[14021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056092, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:32.222127 14037 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:32.222597 14037 webserver.cc:460] Webserver started at http://127.0.0.1:40863/ using document root <none> and password file <none>
I20260806 22:41:32.222893 14037 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:32.222947 14037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:32.223091 14037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:32.224274 14037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "5c41e795f85b476d9e0f09f748e06f5d"
format_stamp: "Formatted at 2026-08-06 22:41:32 on dist-test-slave-3kk6"
server_key: "370797658e5f8a4a66b33b7c56de95a3"
server_key_iv: "80f8e9878cfcbd76dfdb32a5151d0591"
server_key_version: "encryptionkey@0"
I20260806 22:41:32.224665 14037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "5c41e795f85b476d9e0f09f748e06f5d"
format_stamp: "Formatted at 2026-08-06 22:41:32 on dist-test-slave-3kk6"
server_key: "370797658e5f8a4a66b33b7c56de95a3"
server_key_iv: "80f8e9878cfcbd76dfdb32a5151d0591"
server_key_version: "encryptionkey@0"
I20260806 22:41:32.226095 14037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:32.226965 14056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:32.227164 14037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:32.227269 14037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "5c41e795f85b476d9e0f09f748e06f5d"
format_stamp: "Formatted at 2026-08-06 22:41:32 on dist-test-slave-3kk6"
server_key: "370797658e5f8a4a66b33b7c56de95a3"
server_key_iv: "80f8e9878cfcbd76dfdb32a5151d0591"
server_key_version: "encryptionkey@0"
I20260806 22:41:32.227355 14037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:32.244702 14037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:32.246794 14037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:32.246974 14037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:32.251341 14037 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43531
I20260806 22:41:32.251382 14131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43531 every 8 connection(s)
I20260806 22:41:32.251832 14037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:32.252748 14132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:32.254165 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 14037
I20260806 22:41:32.254290 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:32.254521 17325 external_mini_cluster.cc:1428] Setting key 1d2dbd4fa475a0604c9911567cf4bf89
I20260806 22:41:32.255721 14132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d: Bootstrap starting.
I20260806 22:41:32.256484 14132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:32.256793 14132 log.cc:824] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:32.257578 14132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d: No bootstrap required, opened a new log
I20260806 22:41:32.260030 14132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41e795f85b476d9e0f09f748e06f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } }
I20260806 22:41:32.260198 14132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:32.260242 14132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c41e795f85b476d9e0f09f748e06f5d, State: Initialized, Role: FOLLOWER
I20260806 22:41:32.260402 14132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41e795f85b476d9e0f09f748e06f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } }
I20260806 22:41:32.260505 14132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:32.260542 14132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:32.260613 14132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:32.260763 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:32.255141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40155 (local address 127.0.0.1:43531)
0806 22:41:32.255343 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:32.255351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:32.255377 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:41:32.255788 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:32.255794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:32.255870 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:32.256007 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:32.256287 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:32.257435 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:32.258547 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:32.259018 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:32.259111 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:32.259914 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:32.259918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:32.259919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:32.260094 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:32.260126 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:32.260136 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:32.260280 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:32.260420 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 22:41:32.260458 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260806 22:41:32.261348 14132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41e795f85b476d9e0f09f748e06f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } }
I20260806 22:41:32.261493 14132 leader_election.cc:302] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c41e795f85b476d9e0f09f748e06f5d; no voters:
I20260806 22:41:32.261669 14132 leader_election.cc:288] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:32.261693 14138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:32.261826 14138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [term 1 LEADER]: Becoming Leader. State: Replica: 5c41e795f85b476d9e0f09f748e06f5d, State: Running, Role: LEADER
I20260806 22:41:32.261914 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:32.261942 14132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:32.261970 14138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41e795f85b476d9e0f09f748e06f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } }
I20260806 22:41:32.262324 14138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c41e795f85b476d9e0f09f748e06f5d. Latest consensus state: current_term: 1 leader_uuid: "5c41e795f85b476d9e0f09f748e06f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41e795f85b476d9e0f09f748e06f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } } }
I20260806 22:41:32.262334 14140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c41e795f85b476d9e0f09f748e06f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41e795f85b476d9e0f09f748e06f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } } }
I20260806 22:41:32.262418 14138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:32.262455 14140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:32.262639 14145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:32.263383 14145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:32.268191 14145 catalog_manager.cc:1347] Generated new cluster ID: 44b7d939dba54160b6ae149b25a5af9a
I20260806 22:41:32.268255 14145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:32.270821 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:32.267552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46473 (local address 127.0.0.1:43531)
0806 22:41:32.267596 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:32.267598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:32.267747 (+ 149us) server_negotiation.cc:407] Connection header received
0806 22:41:32.267821 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:32.267822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:32.267832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:32.267965 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:32.268166 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:32.268737 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:32.269448 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:32.269678 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:32.269761 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:32.270531 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:32.270532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:32.270532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:32.270537 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:32.270552 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:32.270556 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:32.270610 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:32.270725 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 22:41:32.270735 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:41:32.287305 14145 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:32.287937 14145 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:32.291910 14145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c41e795f85b476d9e0f09f748e06f5d: Generated new TSK 0
I20260806 22:41:32.292138 14145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:32.301975 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 14037
W20260806 22:41:32.312422 14017 connection.cc:535] client connection to 127.0.0.1:43531 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43531): Connection reset by peer (error 104)
2026-08-06T22:41:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1565 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:32 dist-test-slave-3kk6 krb5kdc[14173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:32 dist-test-slave-3kk6 krb5kdc[14173](info): set up 2 sockets
Aug 06 22:41:32 dist-test-slave-3kk6 krb5kdc[14173](info): commencing operation
krb5kdc: starting...
W20260806 22:41:33.526225 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:33 dist-test-slave-3kk6 krb5kdc[14173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:33.657583 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33089
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:33.742686 14190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:33.742951 14190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:33.742992 14190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:33.744175 14190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:33.744248 14190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:33.744277 14190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:33.744310 14190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:33.744335 14190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:33.745874 14190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33089
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:33.746172 14190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:33.746415 14190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:33.751591 14202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:33.751621 14197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:33.751596 14198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:33.751904 14190 server_base.cc:1034] running on GCE node
I20260806 22:41:33.752147 14190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:33.752506 14190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:33.753720 14190 hybrid_clock.cc:648] HybridClock initialized: now 1786056093753700 us; error 40 us; skew 500 ppm
Aug 06 22:41:33 dist-test-slave-3kk6 krb5kdc[14173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056093, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:33.756618 14190 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:33.757218 14190 webserver.cc:460] Webserver started at http://127.0.0.1:36605/ using document root <none> and password file <none>
I20260806 22:41:33.757478 14190 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:33.757529 14190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:33.757732 14190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:33.759060 14190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "eb2fe2a77a8f455093d0924ae33e5060"
format_stamp: "Formatted at 2026-08-06 22:41:33 on dist-test-slave-3kk6"
server_key: "485c4452b2fd655c2446aa870efa9590"
server_key_iv: "01c4b590b23e6d20042962036e9e9c44"
server_key_version: "encryptionkey@0"
I20260806 22:41:33.759480 14190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "eb2fe2a77a8f455093d0924ae33e5060"
format_stamp: "Formatted at 2026-08-06 22:41:33 on dist-test-slave-3kk6"
server_key: "485c4452b2fd655c2446aa870efa9590"
server_key_iv: "01c4b590b23e6d20042962036e9e9c44"
server_key_version: "encryptionkey@0"
I20260806 22:41:33.761009 14190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:41:33.761941 14209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:33.762192 14190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:33.762301 14190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "eb2fe2a77a8f455093d0924ae33e5060"
format_stamp: "Formatted at 2026-08-06 22:41:33 on dist-test-slave-3kk6"
server_key: "485c4452b2fd655c2446aa870efa9590"
server_key_iv: "01c4b590b23e6d20042962036e9e9c44"
server_key_version: "encryptionkey@0"
I20260806 22:41:33.762382 14190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:33.765959 14190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:33.768522 14190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:33.768652 14190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:33.773343 14190 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33089
I20260806 22:41:33.773424 14280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33089 every 8 connection(s)
I20260806 22:41:33.773772 14190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:33.774828 14283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:33.777271 14283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060: Bootstrap starting.
I20260806 22:41:33.777927 14283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:33.778231 14283 log.cc:824] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:33.779120 14283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060: No bootstrap required, opened a new log
I20260806 22:41:33.780794 14283 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2fe2a77a8f455093d0924ae33e5060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33089 } }
I20260806 22:41:33.780953 14283 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:33.780978 14283 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb2fe2a77a8f455093d0924ae33e5060, State: Initialized, Role: FOLLOWER
I20260806 22:41:33.781142 14283 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2fe2a77a8f455093d0924ae33e5060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33089 } }
I20260806 22:41:33.781208 14283 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:33.781250 14283 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:33.781301 14283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:33.781881 14283 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2fe2a77a8f455093d0924ae33e5060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33089 } }
I20260806 22:41:33.781997 14283 leader_election.cc:302] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb2fe2a77a8f455093d0924ae33e5060; no voters:
I20260806 22:41:33.782192 14283 leader_election.cc:288] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:33.782296 14286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:33.782471 14283 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:33.782483 14286 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [term 1 LEADER]: Becoming Leader. State: Replica: eb2fe2a77a8f455093d0924ae33e5060, State: Running, Role: LEADER
I20260806 22:41:33.782699 14286 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2fe2a77a8f455093d0924ae33e5060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33089 } }
I20260806 22:41:33.783325 14290 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb2fe2a77a8f455093d0924ae33e5060. Latest consensus state: current_term: 1 leader_uuid: "eb2fe2a77a8f455093d0924ae33e5060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2fe2a77a8f455093d0924ae33e5060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33089 } } }
I20260806 22:41:33.783421 14290 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:33.783529 14288 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb2fe2a77a8f455093d0924ae33e5060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb2fe2a77a8f455093d0924ae33e5060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33089 } } }
I20260806 22:41:33.783600 14288 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:33.783707 14299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:33.783892 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 14190
I20260806 22:41:33.784019 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:33.784255 17325 external_mini_cluster.cc:1428] Setting key 62766e7898d74f760e6c80ad24d0bfba
I20260806 22:41:33.784310 14299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:33.785956 14299 catalog_manager.cc:1347] Generated new cluster ID: 13ad94fce611451fa75be6c37d53cea5
I20260806 22:41:33.786008 14299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:33.786834 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:33.784999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34039 (local address 127.0.0.1:33089)
0806 22:41:33.785172 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:33.785182 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:33.785217 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:41:33.785337 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:33.785344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:33.785372 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:33.785836 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:33.786363 (+ 527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:33.786366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:33.786366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:33.786369 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:33.786394 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:33.786405 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:33.786483 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:33.786562 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:41:33.786606 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260806 22:41:33.787792 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:33.790390 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:33.789224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44781 (local address 127.0.0.1:33089)
0806 22:41:33.789260 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:33.789262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:33.789390 (+ 128us) server_negotiation.cc:407] Connection header received
0806 22:41:33.789455 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:33.789456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:33.789462 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:33.789517 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:33.790149 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:33.790150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:33.790151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:33.790155 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:33.790171 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:33.790175 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:33.790234 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:33.790290 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:41:33.790305 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:41:33.801239 14299 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:33.801503 14299 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:33.817318 14299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb2fe2a77a8f455093d0924ae33e5060: Generated new TSK 0
I20260806 22:41:33.817560 14299 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:33.821337 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 14190
2026-08-06T22:41:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1526 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:33 dist-test-slave-3kk6 krb5kdc[14327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:33 dist-test-slave-3kk6 krb5kdc[14327](info): set up 2 sockets
Aug 06 22:41:33 dist-test-slave-3kk6 krb5kdc[14327](info): commencing operation
krb5kdc: starting...
W20260806 22:41:35.201416 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:35 dist-test-slave-3kk6 krb5kdc[14327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:35.331669 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:35.418824 14346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:35.419090 14346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:35.419132 14346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:35.420324 14346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:35.420404 14346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:35.420430 14346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:35.420452 14346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:35.420482 14346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:35.422039 14346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:35.422338 14346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:35.422585 14346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:35.427583 14356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:35.427603 14352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:35.427871 14346 server_base.cc:1034] running on GCE node
W20260806 22:41:35.427573 14354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:35.428239 14346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:35.428644 14346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:35.429827 14346 hybrid_clock.cc:648] HybridClock initialized: now 1786056095429813 us; error 36 us; skew 500 ppm
Aug 06 22:41:35 dist-test-slave-3kk6 krb5kdc[14327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056095, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:35.432547 14346 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:35.433066 14346 webserver.cc:460] Webserver started at http://127.0.0.1:39765/ using document root <none> and password file <none>
I20260806 22:41:35.433306 14346 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:35.433378 14346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:35.433544 14346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:35.434747 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "e571db9a96894c1ead9581d441b5b9e5"
format_stamp: "Formatted at 2026-08-06 22:41:35 on dist-test-slave-3kk6"
server_key: "4969baa6ef9b60ee4edb9a3b86434a7c"
server_key_iv: "4fe3251cf3f4a2222cde524fe67ba2c0"
server_key_version: "encryptionkey@0"
I20260806 22:41:35.435171 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "e571db9a96894c1ead9581d441b5b9e5"
format_stamp: "Formatted at 2026-08-06 22:41:35 on dist-test-slave-3kk6"
server_key: "4969baa6ef9b60ee4edb9a3b86434a7c"
server_key_iv: "4fe3251cf3f4a2222cde524fe67ba2c0"
server_key_version: "encryptionkey@0"
I20260806 22:41:35.436573 14346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:41:35.437388 14370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:35.437675 14346 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:35.437788 14346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "e571db9a96894c1ead9581d441b5b9e5"
format_stamp: "Formatted at 2026-08-06 22:41:35 on dist-test-slave-3kk6"
server_key: "4969baa6ef9b60ee4edb9a3b86434a7c"
server_key_iv: "4fe3251cf3f4a2222cde524fe67ba2c0"
server_key_version: "encryptionkey@0"
I20260806 22:41:35.437866 14346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:35.453177 14346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:35.455883 14346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:35.456071 14346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:35.460616 14346 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33313
I20260806 22:41:35.460656 14434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33313 every 8 connection(s)
I20260806 22:41:35.461071 14346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:35.461998 14435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:35.464469 14435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5: Bootstrap starting.
I20260806 22:41:35.465044 14435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:35.465296 14435 log.cc:824] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:35.466028 14435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5: No bootstrap required, opened a new log
I20260806 22:41:35.467643 14435 raft_consensus.cc:348] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e571db9a96894c1ead9581d441b5b9e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260806 22:41:35.467798 14435 raft_consensus.cc:374] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:35.467823 14435 raft_consensus.cc:729] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e571db9a96894c1ead9581d441b5b9e5, State: Initialized, Role: FOLLOWER
I20260806 22:41:35.468010 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 14346
I20260806 22:41:35.468024 14435 consensus_queue.cc:260] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e571db9a96894c1ead9581d441b5b9e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260806 22:41:35.468137 14435 raft_consensus.cc:388] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:35.468153 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:35.468182 14435 raft_consensus.cc:482] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:35.468235 14435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:35.468348 17325 external_mini_cluster.cc:1428] Setting key 6343908cc5b14ac464f1b011ac696056
I20260806 22:41:35.469086 14435 raft_consensus.cc:504] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e571db9a96894c1ead9581d441b5b9e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260806 22:41:35.469204 14435 leader_election.cc:302] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e571db9a96894c1ead9581d441b5b9e5; no voters:
I20260806 22:41:35.469367 14435 leader_election.cc:288] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:35.469446 14446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:35.469620 14446 raft_consensus.cc:686] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [term 1 LEADER]: Becoming Leader. State: Replica: e571db9a96894c1ead9581d441b5b9e5, State: Running, Role: LEADER
I20260806 22:41:35.469645 14435 sys_catalog.cc:564] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:35.469745 14446 consensus_queue.cc:237] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e571db9a96894c1ead9581d441b5b9e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260806 22:41:35.470716 14447 sys_catalog.cc:455] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e571db9a96894c1ead9581d441b5b9e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e571db9a96894c1ead9581d441b5b9e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } } }
I20260806 22:41:35.470829 14447 sys_catalog.cc:458] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:35.470973 14447 sys_catalog.cc:455] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e571db9a96894c1ead9581d441b5b9e5. Latest consensus state: current_term: 1 leader_uuid: "e571db9a96894c1ead9581d441b5b9e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e571db9a96894c1ead9581d441b5b9e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } } }
I20260806 22:41:35.471028 14447 sys_catalog.cc:458] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5 [sys.catalog]: This master's current role is: LEADER
W20260806 22:41:35.471642 14476 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:41:35.471724 14476 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:41:35.471827 14468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:35.471985 14468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:35.473614 14468 catalog_manager.cc:1347] Generated new cluster ID: 8e167c00b8c14a0ab0d5bd6222d749c9
I20260806 22:41:35.473666 14468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:41:35 dist-test-slave-3kk6 krb5kdc[14327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:41:35.482141 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:35.468916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34997 (local address 127.0.0.1:33313)
0806 22:41:35.469135 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:35.469145 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:35.469187 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:41:35.469325 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:35.469333 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:35.469398 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:35.470526 (+ 1128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:35.470628 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:35.472314 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:35.473233 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:35.475057 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:35.475161 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:35.478133 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:35.478138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:35.478139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:35.478156 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:35.480375 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:35.480781 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:35.480785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:35.480785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:35.480927 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:35.481144 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:35.481144 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:35.481145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:35.481496 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:35.481664 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:35.481750 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 22:41:35.481903 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260806 22:41:35.483556 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:35.492509 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:35.485038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33323 (local address 127.0.0.1:33313)
0806 22:41:35.485083 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:35.485085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:35.485133 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:41:35.485210 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:35.485211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:35.485236 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:35.485286 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:35.485966 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:35.486711 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:35.487747 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:35.488048 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:35.488108 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:35.489633 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:35.489634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:35.489634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:35.489641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:35.491286 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:35.491698 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:35.491699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:35.491700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:35.491770 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:35.491980 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:35.491981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:35.491981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:35.492101 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:35.492184 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:35.492339 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 22:41:35.492418 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:41:35.518424 14468 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:35.519243 14468 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:35.531162 14468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e571db9a96894c1ead9581d441b5b9e5: Generated new TSK 0
I20260806 22:41:35.531423 14468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:35.536286 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 14346
2026-08-06T22:41:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1705 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:35 dist-test-slave-3kk6 krb5kdc[14488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:35 dist-test-slave-3kk6 krb5kdc[14488](info): set up 2 sockets
Aug 06 22:41:35 dist-test-slave-3kk6 krb5kdc[14488](info): commencing operation
krb5kdc: starting...
W20260806 22:41:36.790030 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:36 dist-test-slave-3kk6 krb5kdc[14488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:36.922049 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:37.011390 14510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:37.011695 14510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:37.011739 14510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:37.012941 14510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:37.013020 14510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:37.013047 14510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:37.013083 14510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:37.013131 14510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:37.014775 14510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:37.015072 14510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:37.015329 14510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:37.020666 14520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:37.020713 14517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:37.020666 14518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:37.020992 14510 server_base.cc:1034] running on GCE node
I20260806 22:41:37.021287 14510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:37.021637 14510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:37.022779 14510 hybrid_clock.cc:648] HybridClock initialized: now 1786056097022766 us; error 51 us; skew 500 ppm
Aug 06 22:41:37 dist-test-slave-3kk6 krb5kdc[14488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056097, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:37.025549 14510 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:37.026062 14510 webserver.cc:460] Webserver started at http://127.0.0.1:42657/ using document root <none> and password file <none>
I20260806 22:41:37.026285 14510 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:37.026327 14510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:37.026428 14510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:37.027587 14510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "5f1139a37ccd447db4441731734fdf0f"
format_stamp: "Formatted at 2026-08-06 22:41:37 on dist-test-slave-3kk6"
server_key: "608d1f7ed535cfcf1dab5389276067f1"
server_key_iv: "ca15070ada2dc456c3a8df1ad3c991d2"
server_key_version: "encryptionkey@0"
I20260806 22:41:37.027988 14510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "5f1139a37ccd447db4441731734fdf0f"
format_stamp: "Formatted at 2026-08-06 22:41:37 on dist-test-slave-3kk6"
server_key: "608d1f7ed535cfcf1dab5389276067f1"
server_key_iv: "ca15070ada2dc456c3a8df1ad3c991d2"
server_key_version: "encryptionkey@0"
I20260806 22:41:37.029464 14510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:41:37.030224 14528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:37.030458 14510 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:37.030560 14510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "5f1139a37ccd447db4441731734fdf0f"
format_stamp: "Formatted at 2026-08-06 22:41:37 on dist-test-slave-3kk6"
server_key: "608d1f7ed535cfcf1dab5389276067f1"
server_key_iv: "ca15070ada2dc456c3a8df1ad3c991d2"
server_key_version: "encryptionkey@0"
I20260806 22:41:37.030638 14510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:37.052313 14510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:37.054971 14510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:37.055104 14510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:37.059753 14510 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33243
I20260806 22:41:37.059825 14605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33243 every 8 connection(s)
I20260806 22:41:37.060195 14510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:37.061184 14606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:37.063637 14606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f: Bootstrap starting.
I20260806 22:41:37.064278 14606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:37.064587 14606 log.cc:824] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:37.065348 14606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f: No bootstrap required, opened a new log
I20260806 22:41:37.067057 14606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1139a37ccd447db4441731734fdf0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } }
I20260806 22:41:37.067232 14606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:37.067291 14606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f1139a37ccd447db4441731734fdf0f, State: Initialized, Role: FOLLOWER
I20260806 22:41:37.067456 14606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1139a37ccd447db4441731734fdf0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } }
I20260806 22:41:37.067548 14606 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:37.067595 14606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:37.067648 14606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:37.068241 14606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1139a37ccd447db4441731734fdf0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } }
I20260806 22:41:37.068372 14606 leader_election.cc:302] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f1139a37ccd447db4441731734fdf0f; no voters:
I20260806 22:41:37.068583 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 14510
I20260806 22:41:37.068594 14606 leader_election.cc:288] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:37.068629 14609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:37.068720 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:37.068769 14609 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [term 1 LEADER]: Becoming Leader. State: Replica: 5f1139a37ccd447db4441731734fdf0f, State: Running, Role: LEADER
I20260806 22:41:37.068919 14606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:37.068959 17325 external_mini_cluster.cc:1428] Setting key 4aa73554ff1fe5e5378179a30d4a4ddb
I20260806 22:41:37.068922 14609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1139a37ccd447db4441731734fdf0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } }
I20260806 22:41:37.069896 14611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f1139a37ccd447db4441731734fdf0f. Latest consensus state: current_term: 1 leader_uuid: "5f1139a37ccd447db4441731734fdf0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1139a37ccd447db4441731734fdf0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } } }
I20260806 22:41:37.070050 14611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:37.070091 14610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f1139a37ccd447db4441731734fdf0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f1139a37ccd447db4441731734fdf0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33243 } } }
I20260806 22:41:37.070190 14610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:37.070508 14627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:37.071158 14627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:37.073119 14627 catalog_manager.cc:1347] Generated new cluster ID: f413eda83c7a482a853cf37a57c47605
I20260806 22:41:37.073174 14627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:37.076350 14617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:37.069502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49427 (local address 127.0.0.1:33243)
0806 22:41:37.069692 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:37.069704 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:37.070434 (+ 730us) server_negotiation.cc:407] Connection header received
0806 22:41:37.070794 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:37.070804 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:37.070891 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:37.071074 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:37.071591 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:37.072898 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:37.074049 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:37.074680 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:37.074790 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:37.075592 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:37.075597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:37.075597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:37.075820 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:37.075859 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:37.075885 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:37.075957 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:37.076066 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 22:41:37.076130 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260806 22:41:37.077391 14627 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:37.077437 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:37.078167 14627 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:37.081831 14617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:37.078883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36049 (local address 127.0.0.1:33243)
0806 22:41:37.078965 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:37.078967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:37.079017 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:41:37.079093 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:37.079093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:37.079104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:37.079160 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:37.079289 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:37.079810 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:37.080591 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:37.080802 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:37.080889 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:37.081595 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:37.081597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:37.081597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:37.081602 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:37.081615 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:37.081620 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:37.081667 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:37.081724 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:41:37.081751 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:41:37.088594 14627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f1139a37ccd447db4441731734fdf0f: Generated new TSK 0
I20260806 22:41:37.088811 14627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:37.095412 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 14510
2026-08-06T22:41:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:37 dist-test-slave-3kk6 krb5kdc[14652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:37 dist-test-slave-3kk6 krb5kdc[14652](info): set up 2 sockets
Aug 06 22:41:37 dist-test-slave-3kk6 krb5kdc[14652](info): commencing operation
krb5kdc: starting...
W20260806 22:41:38.326838 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:38 dist-test-slave-3kk6 krb5kdc[14652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:38.455140 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:38.542642 14669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:38.542959 14669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:38.543004 14669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:38.544227 14669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:38.544310 14669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:38.544335 14669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:38.544358 14669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:38.544387 14669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:38.545971 14669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:38.546283 14669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:38.546540 14669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:38.551999 14678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:38.552047 14676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:38.552281 14669 server_base.cc:1034] running on GCE node
W20260806 22:41:38.552047 14675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:38.552623 14669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:38.552937 14669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:38.554097 14669 hybrid_clock.cc:648] HybridClock initialized: now 1786056098554073 us; error 42 us; skew 500 ppm
Aug 06 22:41:38 dist-test-slave-3kk6 krb5kdc[14652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:38.559938 14669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:38.560490 14669 webserver.cc:460] Webserver started at http://127.0.0.1:34973/ using document root <none> and password file <none>
I20260806 22:41:38.560737 14669 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:38.560786 14669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:38.560950 14669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:38.562103 14669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "a00a0661c13b41bf8c807bf62d1c8b4a"
format_stamp: "Formatted at 2026-08-06 22:41:38 on dist-test-slave-3kk6"
server_key: "9b61ef7da58402455b207bf9447dce90"
server_key_iv: "84a069be77dc2f8568add31919003c57"
server_key_version: "encryptionkey@0"
I20260806 22:41:38.562503 14669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "a00a0661c13b41bf8c807bf62d1c8b4a"
format_stamp: "Formatted at 2026-08-06 22:41:38 on dist-test-slave-3kk6"
server_key: "9b61ef7da58402455b207bf9447dce90"
server_key_iv: "84a069be77dc2f8568add31919003c57"
server_key_version: "encryptionkey@0"
I20260806 22:41:38.564040 14669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:38.564919 14688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:38.565176 14669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:38.565279 14669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "a00a0661c13b41bf8c807bf62d1c8b4a"
format_stamp: "Formatted at 2026-08-06 22:41:38 on dist-test-slave-3kk6"
server_key: "9b61ef7da58402455b207bf9447dce90"
server_key_iv: "84a069be77dc2f8568add31919003c57"
server_key_version: "encryptionkey@0"
I20260806 22:41:38.565362 14669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:38.568928 14669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:38.571246 14669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:38.571424 14669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:38.576144 14669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39993
I20260806 22:41:38.576198 14761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39993 every 8 connection(s)
I20260806 22:41:38.576571 14669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:38.577737 14763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:38.580492 14763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a: Bootstrap starting.
I20260806 22:41:38.581169 14763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:38.581549 14763 log.cc:824] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:38.581605 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 14669
I20260806 22:41:38.581758 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:38.581979 17325 external_mini_cluster.cc:1428] Setting key b14bc5578fae286f710a51d36e57e4ba
I20260806 22:41:38.582553 14763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a: No bootstrap required, opened a new log
I20260806 22:41:38.585165 14763 raft_consensus.cc:348] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00a0661c13b41bf8c807bf62d1c8b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260806 22:41:38.585448 14763 raft_consensus.cc:374] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:38.585525 14763 raft_consensus.cc:729] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a00a0661c13b41bf8c807bf62d1c8b4a, State: Initialized, Role: FOLLOWER
I20260806 22:41:38.585690 14763 consensus_queue.cc:260] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00a0661c13b41bf8c807bf62d1c8b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260806 22:41:38.585819 14763 raft_consensus.cc:388] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:38.585877 14763 raft_consensus.cc:482] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:38.585893 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:38.582610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41201 (local address 127.0.0.1:39993)
0806 22:41:38.582876 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:38.582888 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:38.582936 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:41:38.584144 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:38.584156 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:38.584202 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:38.584377 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:38.585117 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:38.585121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:38.585122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:38.585127 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:38.585189 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:38.585207 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:38.585295 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:38.585336 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:41:38.585379 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":117,"threads_started":1}
I20260806 22:41:38.585937 14763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:38.586984 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:38.586983 14763 raft_consensus.cc:504] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00a0661c13b41bf8c807bf62d1c8b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260806 22:41:38.587117 14763 leader_election.cc:302] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a00a0661c13b41bf8c807bf62d1c8b4a; no voters:
I20260806 22:41:38.587320 14763 leader_election.cc:288] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:38.587376 14770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:38.587534 14770 raft_consensus.cc:686] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [term 1 LEADER]: Becoming Leader. State: Replica: a00a0661c13b41bf8c807bf62d1c8b4a, State: Running, Role: LEADER
I20260806 22:41:38.587656 14763 sys_catalog.cc:564] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:38.587682 14770 consensus_queue.cc:237] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00a0661c13b41bf8c807bf62d1c8b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260806 22:41:38.588213 14771 sys_catalog.cc:455] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a00a0661c13b41bf8c807bf62d1c8b4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00a0661c13b41bf8c807bf62d1c8b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260806 22:41:38.588305 14771 sys_catalog.cc:458] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:38.588551 14772 sys_catalog.cc:455] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a00a0661c13b41bf8c807bf62d1c8b4a. Latest consensus state: current_term: 1 leader_uuid: "a00a0661c13b41bf8c807bf62d1c8b4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00a0661c13b41bf8c807bf62d1c8b4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260806 22:41:38.588625 14772 sys_catalog.cc:458] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:38.588976 14786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:38.589471 14786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:38.591133 14786 catalog_manager.cc:1347] Generated new cluster ID: c18d489dd3e44e4d8ea83302bbe5b4a2
I20260806 22:41:38.591185 14786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:38.592171 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:38.590523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47341 (local address 127.0.0.1:39993)
0806 22:41:38.590565 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:38.590568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:38.590948 (+ 380us) server_negotiation.cc:407] Connection header received
0806 22:41:38.591039 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:38.591040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:38.591048 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:38.591084 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:38.591904 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:38.591906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:38.591906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:38.591911 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:38.591926 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:38.591931 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:38.591964 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:38.592086 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:41:38.592101 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:41:38.607810 14786 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:38.608036 14786 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:38.615361 14786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a00a0661c13b41bf8c807bf62d1c8b4a: Generated new TSK 0
I20260806 22:41:38.615589 14786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:38.625386 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 14669
2026-08-06T22:41:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:38 dist-test-slave-3kk6 krb5kdc[14814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:38 dist-test-slave-3kk6 krb5kdc[14814](info): set up 2 sockets
Aug 06 22:41:38 dist-test-slave-3kk6 krb5kdc[14814](info): commencing operation
krb5kdc: starting...
W20260806 22:41:39.863056 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:39 dist-test-slave-3kk6 krb5kdc[14814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:40.000367 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:40.087378 14830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:40.087644 14830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:40.087685 14830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:40.088871 14830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:40.088946 14830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:40.088971 14830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:40.088992 14830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:40.089023 14830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:40.090584 14830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:40.090920 14830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:40.091181 14830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:40.096549 14836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:40.096812 14835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:40.096587 14838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:40.097282 14830 server_base.cc:1034] running on GCE node
I20260806 22:41:40.097565 14830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:40.097875 14830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:40.099050 14830 hybrid_clock.cc:648] HybridClock initialized: now 1786056100099025 us; error 43 us; skew 500 ppm
Aug 06 22:41:40 dist-test-slave-3kk6 krb5kdc[14814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:40.102725 14830 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:40.103313 14830 webserver.cc:460] Webserver started at http://127.0.0.1:43761/ using document root <none> and password file <none>
I20260806 22:41:40.103636 14830 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:40.103708 14830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:40.103875 14830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:40.105403 14830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "7dc4832d37ea481381eb9f0aa9728b89"
format_stamp: "Formatted at 2026-08-06 22:41:40 on dist-test-slave-3kk6"
server_key: "75437af9cf34aa7998be29505f8c78ee"
server_key_iv: "ddc73baf77d53f0a1cbf39dd1b5915e6"
server_key_version: "encryptionkey@0"
I20260806 22:41:40.105870 14830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "7dc4832d37ea481381eb9f0aa9728b89"
format_stamp: "Formatted at 2026-08-06 22:41:40 on dist-test-slave-3kk6"
server_key: "75437af9cf34aa7998be29505f8c78ee"
server_key_iv: "ddc73baf77d53f0a1cbf39dd1b5915e6"
server_key_version: "encryptionkey@0"
I20260806 22:41:40.107438 14830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:41:40.108201 14848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:40.108390 14830 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:41:40.108489 14830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "7dc4832d37ea481381eb9f0aa9728b89"
format_stamp: "Formatted at 2026-08-06 22:41:40 on dist-test-slave-3kk6"
server_key: "75437af9cf34aa7998be29505f8c78ee"
server_key_iv: "ddc73baf77d53f0a1cbf39dd1b5915e6"
server_key_version: "encryptionkey@0"
I20260806 22:41:40.108565 14830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:40.125286 14830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:40.127487 14830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:40.127671 14830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:40.132002 14830 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42823
I20260806 22:41:40.132045 14916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42823 every 8 connection(s)
I20260806 22:41:40.132447 14830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:40.133428 14917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:40.135948 14917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89: Bootstrap starting.
I20260806 22:41:40.136591 14917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:40.136904 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 14830
I20260806 22:41:40.136915 14917 log.cc:824] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:40.137042 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:40.137251 17325 external_mini_cluster.cc:1428] Setting key 5f6950d3e51e8053b294037a75a652c4
I20260806 22:41:40.137785 14917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89: No bootstrap required, opened a new log
I20260806 22:41:40.139832 14917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dc4832d37ea481381eb9f0aa9728b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42823 } }
I20260806 22:41:40.140008 14917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:40.140064 14917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dc4832d37ea481381eb9f0aa9728b89, State: Initialized, Role: FOLLOWER
I20260806 22:41:40.140194 14917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dc4832d37ea481381eb9f0aa9728b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42823 } }
I20260806 22:41:40.140259 14917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:40.140282 14917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:40.140347 14917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:40.140985 14917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dc4832d37ea481381eb9f0aa9728b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42823 } }
I20260806 22:41:40.141077 14917 leader_election.cc:302] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dc4832d37ea481381eb9f0aa9728b89; no voters:
I20260806 22:41:40.141301 14917 leader_election.cc:288] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:40.141350 14925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:40.141496 14925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [term 1 LEADER]: Becoming Leader. State: Replica: 7dc4832d37ea481381eb9f0aa9728b89, State: Running, Role: LEADER
I20260806 22:41:40.141588 14917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:40.141638 14925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dc4832d37ea481381eb9f0aa9728b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42823 } }
I20260806 22:41:40.142113 14927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7dc4832d37ea481381eb9f0aa9728b89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dc4832d37ea481381eb9f0aa9728b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42823 } } }
I20260806 22:41:40.142207 14927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:40.142105 14925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dc4832d37ea481381eb9f0aa9728b89. Latest consensus state: current_term: 1 leader_uuid: "7dc4832d37ea481381eb9f0aa9728b89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dc4832d37ea481381eb9f0aa9728b89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42823 } } }
I20260806 22:41:40.142508 14925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:40.142434 14936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:40.143090 14936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:40.145525 14936 catalog_manager.cc:1347] Generated new cluster ID: 1685571cdbf34ef580090edfb3776b3a
I20260806 22:41:40.145586 14936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:41:40 dist-test-slave-3kk6 krb5kdc[14814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:41:40.151336 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:40.137945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:43455 (local address 127.0.0.1:42823)
0806 22:41:40.138168 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:40.138178 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:40.138225 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:41:40.139139 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:40.139147 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:40.139202 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:40.139370 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:40.140053 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:40.141660 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:40.143047 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:40.143716 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:40.143839 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:40.147027 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:40.147031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:40.147031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:40.147045 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:40.149273 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:40.149794 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:40.149798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:40.149799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:40.149932 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:40.150186 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:40.150187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:40.150188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:40.150580 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:40.150815 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:40.150931 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:41:40.151072 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
I20260806 22:41:40.152784 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:40.162130 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:40.154472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:39573 (local address 127.0.0.1:42823)
0806 22:41:40.154512 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:40.154514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:40.154576 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:41:40.154673 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:40.154674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:40.154687 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:40.154749 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:40.155380 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:40.156177 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:40.157180 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:40.157534 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:40.157611 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:40.159129 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:40.159131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:40.159131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:40.159138 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:40.160799 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:40.161239 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:40.161240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:40.161241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:40.161317 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:40.161548 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:40.161549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:40.161550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:40.161693 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:40.161802 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:40.161950 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 22:41:40.162023 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:41:40.163527 14936 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:40.164077 14936 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:40.175083 14936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dc4832d37ea481381eb9f0aa9728b89: Generated new TSK 0
I20260806 22:41:40.175313 14936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:40.180809 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 14830
W20260806 22:41:40.191720 14810 connection.cc:535] client connection to 127.0.0.1:42823 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42823): Connection reset by peer (error 104)
2026-08-06T22:41:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:40 dist-test-slave-3kk6 krb5kdc[14964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:40 dist-test-slave-3kk6 krb5kdc[14964](info): set up 2 sockets
Aug 06 22:41:40 dist-test-slave-3kk6 krb5kdc[14964](info): commencing operation
krb5kdc: starting...
W20260806 22:41:41.410084 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:41 dist-test-slave-3kk6 krb5kdc[14964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:41.546837 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:41.637550 14982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:41.637868 14982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:41.637915 14982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:41.639192 14982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:41.639278 14982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:41.639307 14982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:41.639329 14982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:41.639356 14982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:41.640980 14982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:41.641288 14982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:41.641543 14982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:41.646868 14988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:41.646904 14992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:41.646904 14989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:41.647185 14982 server_base.cc:1034] running on GCE node
I20260806 22:41:41.647428 14982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:41.647770 14982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:41.648947 14982 hybrid_clock.cc:648] HybridClock initialized: now 1786056101648922 us; error 38 us; skew 500 ppm
Aug 06 22:41:41 dist-test-slave-3kk6 krb5kdc[14964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056101, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:41.651734 14982 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:41.652237 14982 webserver.cc:460] Webserver started at http://127.0.0.1:43417/ using document root <none> and password file <none>
I20260806 22:41:41.652504 14982 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:41.652581 14982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:41.652745 14982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:41.653970 14982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "ac7dd1dc51e747e686efe455dc47a01f"
format_stamp: "Formatted at 2026-08-06 22:41:41 on dist-test-slave-3kk6"
server_key: "700235ba4f959cbda5ea777f458a288d"
server_key_iv: "797e813c0a11d65c0abeae54637c6673"
server_key_version: "encryptionkey@0"
I20260806 22:41:41.654384 14982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "ac7dd1dc51e747e686efe455dc47a01f"
format_stamp: "Formatted at 2026-08-06 22:41:41 on dist-test-slave-3kk6"
server_key: "700235ba4f959cbda5ea777f458a288d"
server_key_iv: "797e813c0a11d65c0abeae54637c6673"
server_key_version: "encryptionkey@0"
I20260806 22:41:41.656013 14982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:41:41.656899 14999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:41.657094 14982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:41.657208 14982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "ac7dd1dc51e747e686efe455dc47a01f"
format_stamp: "Formatted at 2026-08-06 22:41:41 on dist-test-slave-3kk6"
server_key: "700235ba4f959cbda5ea777f458a288d"
server_key_iv: "797e813c0a11d65c0abeae54637c6673"
server_key_version: "encryptionkey@0"
I20260806 22:41:41.657291 14982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:41.673883 14982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:41.676590 14982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:41.676774 14982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:41.681214 14982 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37641
I20260806 22:41:41.681288 15072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37641 every 8 connection(s)
I20260806 22:41:41.681684 14982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:41.682761 15073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:41.683521 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 14982
I20260806 22:41:41.683624 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:41.683909 17325 external_mini_cluster.cc:1428] Setting key 5a281f9065bfb6978fc05d556fa002a7
I20260806 22:41:41.686092 15073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f: Bootstrap starting.
I20260806 22:41:41.686862 15073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:41.687199 15073 log.cc:824] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:41.687985 15073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f: No bootstrap required, opened a new log
I20260806 22:41:41.689846 15079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.684524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:38103 (local address 127.0.0.1:37641)
0806 22:41:41.684711 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.684720 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.684748 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:41:41.685189 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.685197 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.685248 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.685368 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:41.685613 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.686555 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.687725 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.688208 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.688319 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.689103 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:41.689108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:41.689108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:41.689307 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:41.689338 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:41.689350 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:41.689418 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.689572 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.689610 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260806 22:41:41.690485 15073 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac7dd1dc51e747e686efe455dc47a01f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } }
I20260806 22:41:41.690738 15073 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:41.690798 15073 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac7dd1dc51e747e686efe455dc47a01f, State: Initialized, Role: FOLLOWER
I20260806 22:41:41.690932 15073 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac7dd1dc51e747e686efe455dc47a01f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } }
I20260806 22:41:41.690976 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:41.691025 15073 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:41.691057 15073 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:41.691103 15073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:41.691964 15073 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac7dd1dc51e747e686efe455dc47a01f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } }
I20260806 22:41:41.692098 15073 leader_election.cc:302] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac7dd1dc51e747e686efe455dc47a01f; no voters:
I20260806 22:41:41.692286 15073 leader_election.cc:288] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:41.692330 15087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:41.692531 15087 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [term 1 LEADER]: Becoming Leader. State: Replica: ac7dd1dc51e747e686efe455dc47a01f, State: Running, Role: LEADER
I20260806 22:41:41.692574 15073 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:41.692704 15087 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac7dd1dc51e747e686efe455dc47a01f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } }
I20260806 22:41:41.693136 15088 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac7dd1dc51e747e686efe455dc47a01f. Latest consensus state: current_term: 1 leader_uuid: "ac7dd1dc51e747e686efe455dc47a01f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac7dd1dc51e747e686efe455dc47a01f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } } }
I20260806 22:41:41.693146 15089 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac7dd1dc51e747e686efe455dc47a01f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac7dd1dc51e747e686efe455dc47a01f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37641 } } }
I20260806 22:41:41.693307 15089 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:41.693295 15088 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f [sys.catalog]: This master's current role is: LEADER
W20260806 22:41:41.694607 15114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:41:41.694741 15114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:41:41.694813 15101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:41.695009 15101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:41.696652 15101 catalog_manager.cc:1347] Generated new cluster ID: e468dc1e2bc44837961aa4c022078031
I20260806 22:41:41.696708 15101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:41.697978 15079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:41.692608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:54943 (local address 127.0.0.1:37641)
0806 22:41:41.692646 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:41.692648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:41.694309 (+ 1661us) server_negotiation.cc:407] Connection header received
0806 22:41:41.694440 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:41.694441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:41.694455 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:41.694520 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:41.694883 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.695607 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.696522 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:41.696830 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:41.696907 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:41.697720 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:41.697721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:41.697722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:41.697727 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:41.697744 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:41.697749 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:41.697820 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:41.697889 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:41:41.697902 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:41:41.708312 15101 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:41.708951 15101 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:41.717310 15101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac7dd1dc51e747e686efe455dc47a01f: Generated new TSK 0
I20260806 22:41:41.717568 15101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:41.725906 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 14982
W20260806 22:41:41.741351 14960 connection.cc:535] client connection to 127.0.0.1:37641 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37641): Connection reset by peer (error 104)
2026-08-06T22:41:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:41 dist-test-slave-3kk6 krb5kdc[15120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:41 dist-test-slave-3kk6 krb5kdc[15120](info): set up 2 sockets
Aug 06 22:41:41 dist-test-slave-3kk6 krb5kdc[15120](info): commencing operation
krb5kdc: starting...
W20260806 22:41:42.954591 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:43 dist-test-slave-3kk6 krb5kdc[15120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:43.090272 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:43.177259 15135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:43.177505 15135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:43.177531 15135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:43.178710 15135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:43.178750 15135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:43.178764 15135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:43.178777 15135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:43.178789 15135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:43.180277 15135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:43.180493 15135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:43.180692 15135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:43.185887 15147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:43.185880 15141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:43.185985 15143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:43.186229 15135 server_base.cc:1034] running on GCE node
I20260806 22:41:43.186486 15135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:43.186829 15135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:43.188027 15135 hybrid_clock.cc:648] HybridClock initialized: now 1786056103188013 us; error 31 us; skew 500 ppm
Aug 06 22:41:43 dist-test-slave-3kk6 krb5kdc[15120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056103, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:43.190804 15135 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:43.191319 15135 webserver.cc:460] Webserver started at http://127.0.0.1:39307/ using document root <none> and password file <none>
I20260806 22:41:43.191578 15135 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:43.191645 15135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:43.191805 15135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:43.192912 15135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "5c1c40c247664f50bbcf817be3d0d611"
format_stamp: "Formatted at 2026-08-06 22:41:43 on dist-test-slave-3kk6"
server_key: "bc0b82c6de81211f65351ff0d651239c"
server_key_iv: "d4fb32adf5880bf86316877162cb7b53"
server_key_version: "encryptionkey@0"
I20260806 22:41:43.193301 15135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "5c1c40c247664f50bbcf817be3d0d611"
format_stamp: "Formatted at 2026-08-06 22:41:43 on dist-test-slave-3kk6"
server_key: "bc0b82c6de81211f65351ff0d651239c"
server_key_iv: "d4fb32adf5880bf86316877162cb7b53"
server_key_version: "encryptionkey@0"
I20260806 22:41:43.194764 15135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:43.195534 15156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:43.195703 15135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:41:43.195803 15135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "5c1c40c247664f50bbcf817be3d0d611"
format_stamp: "Formatted at 2026-08-06 22:41:43 on dist-test-slave-3kk6"
server_key: "bc0b82c6de81211f65351ff0d651239c"
server_key_iv: "d4fb32adf5880bf86316877162cb7b53"
server_key_version: "encryptionkey@0"
I20260806 22:41:43.195884 15135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:43.199358 15135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:43.201376 15135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:43.201541 15135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:43.205883 15135 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41187
I20260806 22:41:43.205922 15224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41187 every 8 connection(s)
I20260806 22:41:43.206290 15135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:43.207352 15225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:43.209712 15225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611: Bootstrap starting.
I20260806 22:41:43.210280 15225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:43.210547 15225 log.cc:824] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:43.211321 15225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611: No bootstrap required, opened a new log
I20260806 22:41:43.212934 15225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1c40c247664f50bbcf817be3d0d611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } }
I20260806 22:41:43.213083 15225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:43.213106 15225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1c40c247664f50bbcf817be3d0d611, State: Initialized, Role: FOLLOWER
I20260806 22:41:43.213214 15225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1c40c247664f50bbcf817be3d0d611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } }
I20260806 22:41:43.213272 15225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:43.213294 15225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:43.213320 15225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:43.213903 15225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1c40c247664f50bbcf817be3d0d611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } }
I20260806 22:41:43.213999 15225 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c1c40c247664f50bbcf817be3d0d611; no voters:
I20260806 22:41:43.214136 15225 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:43.214196 15230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:43.214421 15230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [term 1 LEADER]: Becoming Leader. State: Replica: 5c1c40c247664f50bbcf817be3d0d611, State: Running, Role: LEADER
I20260806 22:41:43.214473 15225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:43.214524 15230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1c40c247664f50bbcf817be3d0d611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } }
I20260806 22:41:43.215006 15232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1c40c247664f50bbcf817be3d0d611. Latest consensus state: current_term: 1 leader_uuid: "5c1c40c247664f50bbcf817be3d0d611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1c40c247664f50bbcf817be3d0d611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } } }
I20260806 22:41:43.215106 15232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:43.215255 15231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c1c40c247664f50bbcf817be3d0d611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1c40c247664f50bbcf817be3d0d611" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41187 } } }
I20260806 22:41:43.215369 15240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:43.215380 15231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:43.215999 15240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:43.216388 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 15135
I20260806 22:41:43.216533 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:43.216744 17325 external_mini_cluster.cc:1428] Setting key 9621a8ecf4ab0b354f1f35dafc7b09b6
I20260806 22:41:43.219110 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:43.217306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:54107 (local address 127.0.0.1:41187)
0806 22:41:43.217492 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:43.217503 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:43.217542 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:41:43.217658 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:43.217663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:43.217708 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:43.217849 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:43.218636 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:43.218640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:43.218640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:43.218644 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:43.218712 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:43.218727 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:43.218804 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:43.218886 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:41:43.218935 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260806 22:41:43.220124 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:43.222613 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:43.221530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:33073 (local address 127.0.0.1:41187)
0806 22:41:43.221576 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:43.221578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:43.221671 (+ 93us) server_negotiation.cc:407] Connection header received
0806 22:41:43.221716 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:43.221717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:43.221724 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:43.221787 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:43.222396 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:43.222397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:43.222397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:43.222402 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:43.222419 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:43.222423 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:43.222492 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:43.222531 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 22:41:43.222546 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:41:43.223943 15240 catalog_manager.cc:1347] Generated new cluster ID: 2a6cb420e8ed4d9c8594445a7bb45ada
I20260806 22:41:43.224001 15240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:43.238816 15240 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:43.239022 15240 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:43.246891 15240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c1c40c247664f50bbcf817be3d0d611: Generated new TSK 0
I20260806 22:41:43.247133 15240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:43.251145 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 15135
2026-08-06T22:41:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:43 dist-test-slave-3kk6 krb5kdc[15263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:43 dist-test-slave-3kk6 krb5kdc[15263](info): set up 2 sockets
Aug 06 22:41:43 dist-test-slave-3kk6 krb5kdc[15263](info): commencing operation
krb5kdc: starting...
W20260806 22:41:44.486881 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:44 dist-test-slave-3kk6 krb5kdc[15263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:44.623366 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46223
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:44.710865 15281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:44.711109 15281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:44.711153 15281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:44.712344 15281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:44.712417 15281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:44.712445 15281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:44.712476 15281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:44.712507 15281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:44.714046 15281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46223
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:44.714339 15281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:44.714577 15281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:44.719547 15286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:44.719558 15290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:44.719551 15288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:44.719779 15281 server_base.cc:1034] running on GCE node
I20260806 22:41:44.720014 15281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:44.720302 15281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:44.721473 15281 hybrid_clock.cc:648] HybridClock initialized: now 1786056104721461 us; error 30 us; skew 500 ppm
Aug 06 22:41:44 dist-test-slave-3kk6 krb5kdc[15263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056104, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:44.727949 15281 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:44.728433 15281 webserver.cc:460] Webserver started at http://127.0.0.1:46187/ using document root <none> and password file <none>
I20260806 22:41:44.728670 15281 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:44.728745 15281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:44.728894 15281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:44.730053 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "3c507f3d5b2c45419c9ac35a5b3162e7"
format_stamp: "Formatted at 2026-08-06 22:41:44 on dist-test-slave-3kk6"
server_key: "da3e6016549e08e8d89d4bf634cd276f"
server_key_iv: "0ffe6d160c62adcb571da7d9c8c155e7"
server_key_version: "encryptionkey@0"
I20260806 22:41:44.730448 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "3c507f3d5b2c45419c9ac35a5b3162e7"
format_stamp: "Formatted at 2026-08-06 22:41:44 on dist-test-slave-3kk6"
server_key: "da3e6016549e08e8d89d4bf634cd276f"
server_key_iv: "0ffe6d160c62adcb571da7d9c8c155e7"
server_key_version: "encryptionkey@0"
I20260806 22:41:44.732012 15281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:44.732772 15297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:44.732952 15281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:41:44.733059 15281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "3c507f3d5b2c45419c9ac35a5b3162e7"
format_stamp: "Formatted at 2026-08-06 22:41:44 on dist-test-slave-3kk6"
server_key: "da3e6016549e08e8d89d4bf634cd276f"
server_key_iv: "0ffe6d160c62adcb571da7d9c8c155e7"
server_key_version: "encryptionkey@0"
I20260806 22:41:44.733134 15281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:44.744184 15281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:44.746315 15281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:44.746485 15281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:44.750770 15281 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46223
I20260806 22:41:44.750816 15375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46223 every 8 connection(s)
I20260806 22:41:44.751199 15281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:44.752103 15377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:44.754503 15377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7: Bootstrap starting.
I20260806 22:41:44.755133 15377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:44.755398 15377 log.cc:824] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:44.756108 15377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7: No bootstrap required, opened a new log
I20260806 22:41:44.757684 15377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c507f3d5b2c45419c9ac35a5b3162e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46223 } }
I20260806 22:41:44.757845 15377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:44.757869 15377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c507f3d5b2c45419c9ac35a5b3162e7, State: Initialized, Role: FOLLOWER
I20260806 22:41:44.757977 15377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c507f3d5b2c45419c9ac35a5b3162e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46223 } }
I20260806 22:41:44.758046 15377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:44.758090 15377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:44.758128 15377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:44.758770 15377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c507f3d5b2c45419c9ac35a5b3162e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46223 } }
I20260806 22:41:44.758958 15377 leader_election.cc:302] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c507f3d5b2c45419c9ac35a5b3162e7; no voters:
I20260806 22:41:44.759191 15377 leader_election.cc:288] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:44.759322 15380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:44.759498 15380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [term 1 LEADER]: Becoming Leader. State: Replica: 3c507f3d5b2c45419c9ac35a5b3162e7, State: Running, Role: LEADER
I20260806 22:41:44.759639 15380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c507f3d5b2c45419c9ac35a5b3162e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46223 } }
I20260806 22:41:44.759769 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 15281
I20260806 22:41:44.759884 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:44.760035 15384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c507f3d5b2c45419c9ac35a5b3162e7. Latest consensus state: current_term: 1 leader_uuid: "3c507f3d5b2c45419c9ac35a5b3162e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c507f3d5b2c45419c9ac35a5b3162e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46223 } } }
I20260806 22:41:44.760087 17325 external_mini_cluster.cc:1428] Setting key f0144a3c7eb422c2f2b761dc1ee70d45
I20260806 22:41:44.760130 15384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:44.760190 15382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c507f3d5b2c45419c9ac35a5b3162e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c507f3d5b2c45419c9ac35a5b3162e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46223 } } }
I20260806 22:41:44.760279 15382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:44.759495 15377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 22:41:44.762423 15413 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:41:44.762493 15413 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:41:44.762558 15391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:44.762753 15391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:44.764616 15391 catalog_manager.cc:1347] Generated new cluster ID: b688bca328394a4283006780ec49565e
I20260806 22:41:44.764673 15391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:41:44 dist-test-slave-3kk6 krb5kdc[15263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:41:44.772598 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:44.760647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:46211 (local address 127.0.0.1:46223)
0806 22:41:44.760872 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:44.760881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:44.760918 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:41:44.761033 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:44.761038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:44.761083 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:44.761222 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:44.761901 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:44.763423 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:44.764451 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:44.765256 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:44.765376 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:44.768399 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:44.768404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:44.768404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:44.768417 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:44.770438 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:44.770923 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:44.770927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:44.770927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:44.771045 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:44.771262 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:44.771263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:44.771263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:44.771581 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:44.771752 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:44.772238 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 22:41:44.772346 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
I20260806 22:41:44.773747 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:44.782948 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:44.775329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:33393 (local address 127.0.0.1:46223)
0806 22:41:44.775366 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:44.775367 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:44.775493 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:41:44.775559 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:44.775560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:44.775571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:44.775624 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:44.776336 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:44.777131 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:44.778156 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:44.778483 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:44.778544 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:44.780071 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:44.780072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:44.780072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:44.780079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:44.781690 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:44.782123 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:44.782124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:44.782124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:44.782180 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:44.782416 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:44.782416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:44.782417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:44.782534 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:44.782619 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:44.782795 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 22:41:44.782856 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:41:44.796846 15391 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:44.797405 15391 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:44.800499 15391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c507f3d5b2c45419c9ac35a5b3162e7: Generated new TSK 0
I20260806 22:41:44.800702 15391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:44.803606 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 15281
W20260806 22:41:44.816553 15261 connection.cc:535] client connection to 127.0.0.1:46223 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46223): Connection reset by peer (error 104)
2026-08-06T22:41:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:44 dist-test-slave-3kk6 krb5kdc[15424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:44 dist-test-slave-3kk6 krb5kdc[15424](info): set up 2 sockets
Aug 06 22:41:44 dist-test-slave-3kk6 krb5kdc[15424](info): commencing operation
krb5kdc: starting...
W20260806 22:41:46.033620 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:46 dist-test-slave-3kk6 krb5kdc[15424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:46.161854 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40535
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:46.246049 15441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:46.246279 15441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:46.246304 15441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:46.247488 15441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:46.247535 15441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:46.247550 15441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:46.247601 15441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:46.247627 15441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:46.249130 15441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40535
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:46.249397 15441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:46.249634 15441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:46.254580 15447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:46.254597 15446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:46.254634 15441 server_base.cc:1034] running on GCE node
W20260806 22:41:46.254597 15449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:46.255092 15441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:46.255383 15441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:46.256518 15441 hybrid_clock.cc:648] HybridClock initialized: now 1786056106256497 us; error 41 us; skew 500 ppm
Aug 06 22:41:46 dist-test-slave-3kk6 krb5kdc[15424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:46.259539 15441 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:46.260135 15441 webserver.cc:460] Webserver started at http://127.0.0.1:38673/ using document root <none> and password file <none>
I20260806 22:41:46.260443 15441 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:46.260515 15441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:46.260677 15441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:46.262213 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "6c8cd333ea4743848e067dd4d2669426"
format_stamp: "Formatted at 2026-08-06 22:41:46 on dist-test-slave-3kk6"
server_key: "ec57860fb72928a260bb17fa6b59294f"
server_key_iv: "b0fadfc67fe3eab90b2470729305e7db"
server_key_version: "encryptionkey@0"
I20260806 22:41:46.262699 15441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "6c8cd333ea4743848e067dd4d2669426"
format_stamp: "Formatted at 2026-08-06 22:41:46 on dist-test-slave-3kk6"
server_key: "ec57860fb72928a260bb17fa6b59294f"
server_key_iv: "b0fadfc67fe3eab90b2470729305e7db"
server_key_version: "encryptionkey@0"
I20260806 22:41:46.265154 15441 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260806 22:41:46.266067 15456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:46.266279 15441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:46.266389 15441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "6c8cd333ea4743848e067dd4d2669426"
format_stamp: "Formatted at 2026-08-06 22:41:46 on dist-test-slave-3kk6"
server_key: "ec57860fb72928a260bb17fa6b59294f"
server_key_iv: "b0fadfc67fe3eab90b2470729305e7db"
server_key_version: "encryptionkey@0"
I20260806 22:41:46.266486 15441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:46.277328 15441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:46.279747 15441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:46.279917 15441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:46.284108 15441 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40535
I20260806 22:41:46.284142 15533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40535 every 8 connection(s)
I20260806 22:41:46.284513 15441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:46.285419 15534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:46.287369 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 15441
I20260806 22:41:46.287489 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:46.287695 17325 external_mini_cluster.cc:1428] Setting key c67dac259d0302884a913dd041730365
I20260806 22:41:46.288101 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426: Bootstrap starting.
I20260806 22:41:46.288883 15534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:46.289225 15534 log.cc:824] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:46.290100 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426: No bootstrap required, opened a new log
I20260806 22:41:46.292493 15534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8cd333ea4743848e067dd4d2669426" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40535 } }
I20260806 22:41:46.292685 15534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:46.292727 15534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8cd333ea4743848e067dd4d2669426, State: Initialized, Role: FOLLOWER
I20260806 22:41:46.292866 15534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8cd333ea4743848e067dd4d2669426" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40535 } }
I20260806 22:41:46.292948 15534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:46.292971 15534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:46.293048 15534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:46.293767 15534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8cd333ea4743848e067dd4d2669426" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40535 } }
I20260806 22:41:46.293885 15534 leader_election.cc:302] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c8cd333ea4743848e067dd4d2669426; no voters:
I20260806 22:41:46.294073 15534 leader_election.cc:288] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:46.294215 15540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:46.294397 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:46.288298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:42671 (local address 127.0.0.1:40535)
0806 22:41:46.288506 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:46.288518 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:46.288557 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:41:46.289219 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:46.289229 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:46.289296 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:46.289470 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:46.289694 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:46.291122 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:46.291926 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:46.292596 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:46.292719 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:46.293563 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:46.293567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:46.293568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:46.293797 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:46.293839 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:46.293855 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:46.293958 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:46.294078 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:41:46.294111 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260806 22:41:46.294377 15540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [term 1 LEADER]: Becoming Leader. State: Replica: 6c8cd333ea4743848e067dd4d2669426, State: Running, Role: LEADER
I20260806 22:41:46.294531 15540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8cd333ea4743848e067dd4d2669426" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40535 } }
I20260806 22:41:46.294390 15534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:46.295500 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:46.295734 15542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c8cd333ea4743848e067dd4d2669426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8cd333ea4743848e067dd4d2669426" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40535 } } }
I20260806 22:41:46.295835 15542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:46.296139 15557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:46.296716 15557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:46.296978 15543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c8cd333ea4743848e067dd4d2669426. Latest consensus state: current_term: 1 leader_uuid: "6c8cd333ea4743848e067dd4d2669426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8cd333ea4743848e067dd4d2669426" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40535 } } }
I20260806 22:41:46.297060 15543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:46.298349 15557 catalog_manager.cc:1347] Generated new cluster ID: 43dff96ede2042d2874e59fb170c8c83
I20260806 22:41:46.298401 15557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:46.306332 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:46.302762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:34575 (local address 127.0.0.1:40535)
0806 22:41:46.302820 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:46.302822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:46.302834 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:41:46.302860 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:46.302861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:46.302872 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:46.302943 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:46.303124 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:46.303901 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:46.304905 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:46.305208 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:46.305299 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:46.306058 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:46.306059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:46.306060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:46.306065 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:46.306082 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:46.306087 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:46.306157 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:46.306230 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 22:41:46.306241 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:41:46.314167 15557 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:46.314816 15557 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:46.319916 15557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c8cd333ea4743848e067dd4d2669426: Generated new TSK 0
I20260806 22:41:46.320125 15557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:46.328766 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 15441
W20260806 22:41:46.341544 15422 connection.cc:535] client connection to 127.0.0.1:40535 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40535): Connection reset by peer (error 104)
2026-08-06T22:41:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:46 dist-test-slave-3kk6 krb5kdc[15573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:46 dist-test-slave-3kk6 krb5kdc[15573](info): set up 2 sockets
Aug 06 22:41:46 dist-test-slave-3kk6 krb5kdc[15573](info): commencing operation
krb5kdc: starting...
W20260806 22:41:47.555186 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:47 dist-test-slave-3kk6 krb5kdc[15573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:41:47.689718 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35679
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:47.777293 15591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:47.777519 15591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:47.777542 15591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:47.778743 15591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:47.778800 15591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:47.778816 15591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:47.778829 15591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:47.778867 15591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:47.780428 15591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35679
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:47.780699 15591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:47.780951 15591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:47.785951 15600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:47.786011 15601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:47.785948 15605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:47.786051 15591 server_base.cc:1034] running on GCE node
I20260806 22:41:47.786386 15591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:47.786746 15591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:47.787915 15591 hybrid_clock.cc:648] HybridClock initialized: now 1786056107787896 us; error 40 us; skew 500 ppm
Aug 06 22:41:47 dist-test-slave-3kk6 krb5kdc[15573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056107, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:47.790572 15591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:41:47.791085 15591 webserver.cc:460] Webserver started at http://127.0.0.1:40119/ using document root <none> and password file <none>
I20260806 22:41:47.791319 15591 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:47.791368 15591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:47.791535 15591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:47.792672 15591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "e1ef81de67a64f78a4f51ecedc60d6ff"
format_stamp: "Formatted at 2026-08-06 22:41:47 on dist-test-slave-3kk6"
server_key: "c54ed9f29a6ea92ec1d55ee99e75f36b"
server_key_iv: "367ceb1a1a5e48f664d69cf74e3d8f97"
server_key_version: "encryptionkey@0"
I20260806 22:41:47.793056 15591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "e1ef81de67a64f78a4f51ecedc60d6ff"
format_stamp: "Formatted at 2026-08-06 22:41:47 on dist-test-slave-3kk6"
server_key: "c54ed9f29a6ea92ec1d55ee99e75f36b"
server_key_iv: "367ceb1a1a5e48f664d69cf74e3d8f97"
server_key_version: "encryptionkey@0"
I20260806 22:41:47.794484 15591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:47.795377 15614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:47.795609 15591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:47.795696 15591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "e1ef81de67a64f78a4f51ecedc60d6ff"
format_stamp: "Formatted at 2026-08-06 22:41:47 on dist-test-slave-3kk6"
server_key: "c54ed9f29a6ea92ec1d55ee99e75f36b"
server_key_iv: "367ceb1a1a5e48f664d69cf74e3d8f97"
server_key_version: "encryptionkey@0"
I20260806 22:41:47.795801 15591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:47.799216 15591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:47.801429 15591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:47.801728 15591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:47.806317 15591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35679
I20260806 22:41:47.806490 15686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35679 every 8 connection(s)
I20260806 22:41:47.806773 15591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:47.807816 15687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:47.810217 15687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff: Bootstrap starting.
I20260806 22:41:47.810882 15687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:47.811156 15687 log.cc:824] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:47.811908 15687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff: No bootstrap required, opened a new log
I20260806 22:41:47.813522 15687 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ef81de67a64f78a4f51ecedc60d6ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35679 } }
I20260806 22:41:47.813670 15687 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:47.813695 15687 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ef81de67a64f78a4f51ecedc60d6ff, State: Initialized, Role: FOLLOWER
I20260806 22:41:47.813808 15687 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ef81de67a64f78a4f51ecedc60d6ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35679 } }
I20260806 22:41:47.813901 15687 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:47.813941 15687 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:47.814013 15687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:47.814604 15687 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ef81de67a64f78a4f51ecedc60d6ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35679 } }
I20260806 22:41:47.814761 15687 leader_election.cc:302] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1ef81de67a64f78a4f51ecedc60d6ff; no voters:
I20260806 22:41:47.814957 15687 leader_election.cc:288] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:47.815032 15690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:47.815222 15690 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [term 1 LEADER]: Becoming Leader. State: Replica: e1ef81de67a64f78a4f51ecedc60d6ff, State: Running, Role: LEADER
I20260806 22:41:47.815289 15687 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:47.815358 15690 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ef81de67a64f78a4f51ecedc60d6ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35679 } }
I20260806 22:41:47.815742 15691 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1ef81de67a64f78a4f51ecedc60d6ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ef81de67a64f78a4f51ecedc60d6ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35679 } } }
I20260806 22:41:47.815855 15691 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:47.815737 15692 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1ef81de67a64f78a4f51ecedc60d6ff. Latest consensus state: current_term: 1 leader_uuid: "e1ef81de67a64f78a4f51ecedc60d6ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1ef81de67a64f78a4f51ecedc60d6ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35679 } } }
I20260806 22:41:47.816018 15692 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:47.816154 15697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:47.816360 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 15591
I20260806 22:41:47.816498 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:47.816712 17325 external_mini_cluster.cc:1428] Setting key ef64f3d8b0448304ebff74c3b45fd941
I20260806 22:41:47.817016 15697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:47.818749 15697 catalog_manager.cc:1347] Generated new cluster ID: d5394bc4ef254a4884255e0bd019e5fb
I20260806 22:41:47.818806 15697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:47.820384 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:47.817463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:44893 (local address 127.0.0.1:35679)
0806 22:41:47.817659 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:47.817670 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:47.818900 (+ 1230us) server_negotiation.cc:407] Connection header received
0806 22:41:47.819043 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:47.819055 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:47.819079 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:47.819215 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:47.819957 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:47.819961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:47.819961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:47.819965 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:47.820014 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:47.820025 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:47.820088 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:47.820171 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:41:47.820217 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260806 22:41:47.821360 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:47.824004 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:47.822782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:46985 (local address 127.0.0.1:35679)
0806 22:41:47.822820 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:47.822822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:47.822984 (+ 162us) server_negotiation.cc:407] Connection header received
0806 22:41:47.823064 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:47.823065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:47.823071 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:47.823126 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:47.823776 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:47.823777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:47.823777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:47.823781 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:47.823795 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:47.823800 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:47.823850 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:47.823904 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:41:47.823919 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:41:47.843351 15697 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:47.843572 15697 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:47.852083 15697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1ef81de67a64f78a4f51ecedc60d6ff: Generated new TSK 0
I20260806 22:41:47.852319 15697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:47.857033 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 15591
2026-08-06T22:41:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:47 dist-test-slave-3kk6 krb5kdc[15731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:47 dist-test-slave-3kk6 krb5kdc[15731](info): set up 2 sockets
Aug 06 22:41:47 dist-test-slave-3kk6 krb5kdc[15731](info): commencing operation
krb5kdc: starting...
W20260806 22:41:49.080878 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:49 dist-test-slave-3kk6 krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:49.214476 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33945
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:41217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:33945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:49.301456 15747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:49.301754 15747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:49.301797 15747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:49.303081 15747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:49.303148 15747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:49.303172 15747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:49.303195 15747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:49.303227 15747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:49.304813 15747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:41217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:33945
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:33945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:49.305110 15747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:49.305373 15747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:41:49.310714 15747 server_base.cc:1034] running on GCE node
W20260806 22:41:49.310684 15754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:49.310693 15755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:49.310675 15758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:49.311184 15747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:49.311512 15747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:49.312688 15747 hybrid_clock.cc:648] HybridClock initialized: now 1786056109312677 us; error 36 us; skew 500 ppm
Aug 06 22:41:49 dist-test-slave-3kk6 krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056109, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:49.315464 15747 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:49.315932 15747 webserver.cc:460] Webserver started at http://127.16.235.126:46577/ using document root <none> and password file <none>
I20260806 22:41:49.316183 15747 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:49.316231 15747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:49.316406 15747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:49.317554 15747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1"
format_stamp: "Formatted at 2026-08-06 22:41:49 on dist-test-slave-3kk6"
server_key: "92592bc3104ae3edf6c0cc42bcb879c7"
server_key_iv: "596f55f015284ffc48ebea538190d763"
server_key_version: "encryptionkey@0"
I20260806 22:41:49.317945 15747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1"
format_stamp: "Formatted at 2026-08-06 22:41:49 on dist-test-slave-3kk6"
server_key: "92592bc3104ae3edf6c0cc42bcb879c7"
server_key_iv: "596f55f015284ffc48ebea538190d763"
server_key_version: "encryptionkey@0"
I20260806 22:41:49.319489 15747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:41:49.320245 15766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:49.320433 15747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:49.320549 15747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1"
format_stamp: "Formatted at 2026-08-06 22:41:49 on dist-test-slave-3kk6"
server_key: "92592bc3104ae3edf6c0cc42bcb879c7"
server_key_iv: "596f55f015284ffc48ebea538190d763"
server_key_version: "encryptionkey@0"
I20260806 22:41:49.320631 15747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:49.340229 15747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:49.342918 15747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:49.343209 15747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:49.347563 15747 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:33945
I20260806 22:41:49.347621 15841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:33945 every 8 connection(s)
I20260806 22:41:49.348058 15747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:49.349077 15842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:49.350991 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 15747
I20260806 22:41:49.351138 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:49.351361 17325 external_mini_cluster.cc:1428] Setting key b87301e93a60c9c7dceae668969253ed
I20260806 22:41:49.351989 15842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1: Bootstrap starting.
I20260806 22:41:49.352800 15842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:49.353147 15842 log.cc:824] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:49.353936 15842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1: No bootstrap required, opened a new log
I20260806 22:41:49.356380 15842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33945 } }
I20260806 22:41:49.356575 15842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:49.356611 15842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2c8ecc84684079a8bcbdb8d1f73aa1, State: Initialized, Role: FOLLOWER
I20260806 22:41:49.356755 15842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33945 } }
I20260806 22:41:49.356849 15842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:49.356894 15842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:49.356937 15842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:49.357734 15842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33945 } }
I20260806 22:41:49.357836 15842 leader_election.cc:302] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2c8ecc84684079a8bcbdb8d1f73aa1; no voters:
I20260806 22:41:49.358045 15842 leader_election.cc:288] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:49.358103 15848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:49.358300 15848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2c8ecc84684079a8bcbdb8d1f73aa1, State: Running, Role: LEADER
I20260806 22:41:49.358376 15842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:49.358451 15848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33945 } }
I20260806 22:41:49.358867 15850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33945 } } }
I20260806 22:41:49.358846 15848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b2c8ecc84684079a8bcbdb8d1f73aa1. Latest consensus state: current_term: 1 leader_uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2c8ecc84684079a8bcbdb8d1f73aa1" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 33945 } } }
I20260806 22:41:49.358963 15850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:49.358966 15848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1 [sys.catalog]: This master's current role is: LEADER
Aug 06 22:41:49 dist-test-slave-3kk6 krb5kdc[15731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:49.359745 15859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:49.360337 15859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:49.362038 15859 catalog_manager.cc:1347] Generated new cluster ID: 8b66cd5c48494277b066992156c8210d
I20260806 22:41:49.362094 15859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:49.364524 15846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:49.352018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42765 (local address 127.16.235.126:33945)
0806 22:41:49.352231 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:49.352240 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:49.352275 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:41:49.352946 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:49.352957 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:49.353033 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:49.353212 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:49.353873 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:49.355368 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:49.356156 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:49.356786 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:49.356909 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:49.360204 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:49.360209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:49.360209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:49.360224 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:49.362383 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:49.362955 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:49.362959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:49.362959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:49.363071 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:49.363363 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:49.363363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:49.363364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:49.363675 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:49.363830 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:49.364171 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 22:41:49.364301 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
I20260806 22:41:49.365888 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:49.372006 15859 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:49.372560 15859 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:49.375622 15846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:49.367479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56013 (local address 127.16.235.126:33945)
0806 22:41:49.367520 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:49.367522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:49.367640 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:41:49.367738 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:49.367739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:49.367750 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:49.367779 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:49.368609 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:49.369408 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:49.370488 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:49.370836 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:49.370884 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:49.372527 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:49.372528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:49.372529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:49.372536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:49.374169 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:49.374674 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:49.374675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:49.374675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:49.374738 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:49.374991 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:49.374991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:49.374992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:49.375117 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:49.375207 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:49.375458 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 22:41:49.375528 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:41:49.383260 15859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b2c8ecc84684079a8bcbdb8d1f73aa1: Generated new TSK 0
I20260806 22:41:49.383496 15859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:49.387532 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 15747
W20260806 22:41:49.406189 15728 connection.cc:535] client connection to 127.16.235.126:33945 recv error: Network error: failed to read from TLS socket (remote: 127.16.235.126:33945): Connection reset by peer (error 104)
2026-08-06T22:41:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1539 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:49 dist-test-slave-3kk6 krb5kdc[15883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:49 dist-test-slave-3kk6 krb5kdc[15883](info): set up 2 sockets
Aug 06 22:41:49 dist-test-slave-3kk6 krb5kdc[15883](info): commencing operation
krb5kdc: starting...
W20260806 22:41:50.617902 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:50 dist-test-slave-3kk6 krb5kdc[15883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:50.754060 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36643
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:41779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:36643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:50.844177 15902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:50.844468 15902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:50.844514 15902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:50.845811 15902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:50.845890 15902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:50.845916 15902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:50.845938 15902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:50.845971 15902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:50.847652 15902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:41779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:36643
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:36643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:50.848129 15902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:50.848620 15902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:50.854058 15907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:50.854061 15910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:50.854146 15902 server_base.cc:1034] running on GCE node
W20260806 22:41:50.854061 15908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:50.854578 15902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:50.854954 15902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:50.856127 15902 hybrid_clock.cc:648] HybridClock initialized: now 1786056110856104 us; error 45 us; skew 500 ppm
Aug 06 22:41:50 dist-test-slave-3kk6 krb5kdc[15883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056110, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:50.862550 15902 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:50.863095 15902 webserver.cc:460] Webserver started at http://127.16.235.126:35471/ using document root <none> and password file <none>
I20260806 22:41:50.863337 15902 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:50.863407 15902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:50.863564 15902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:50.864729 15902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "8abec542746046609776b7ec1e80cb4b"
format_stamp: "Formatted at 2026-08-06 22:41:50 on dist-test-slave-3kk6"
server_key: "3f95244e46605f1a9477ff22f55abb39"
server_key_iv: "03259a9a50a8dad86cd20e82c76374e3"
server_key_version: "encryptionkey@0"
I20260806 22:41:50.865129 15902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "8abec542746046609776b7ec1e80cb4b"
format_stamp: "Formatted at 2026-08-06 22:41:50 on dist-test-slave-3kk6"
server_key: "3f95244e46605f1a9477ff22f55abb39"
server_key_iv: "03259a9a50a8dad86cd20e82c76374e3"
server_key_version: "encryptionkey@0"
I20260806 22:41:50.866559 15902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:50.867465 15918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:50.867676 15902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:50.867789 15902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "8abec542746046609776b7ec1e80cb4b"
format_stamp: "Formatted at 2026-08-06 22:41:50 on dist-test-slave-3kk6"
server_key: "3f95244e46605f1a9477ff22f55abb39"
server_key_iv: "03259a9a50a8dad86cd20e82c76374e3"
server_key_version: "encryptionkey@0"
I20260806 22:41:50.867888 15902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:50.888974 15902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:50.891276 15902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:50.891460 15902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:50.895745 15902 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:36643
I20260806 22:41:50.895810 15987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:36643 every 8 connection(s)
I20260806 22:41:50.896209 15902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:50.897188 15989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:50.899621 15989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b: Bootstrap starting.
I20260806 22:41:50.900218 15989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:50.900470 15989 log.cc:824] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:50.900861 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 15902
I20260806 22:41:50.901013 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:50.901238 17325 external_mini_cluster.cc:1428] Setting key 15bf0e646c4a7530be5dd508df709113
I20260806 22:41:50.901348 15989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b: No bootstrap required, opened a new log
I20260806 22:41:50.903849 15989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abec542746046609776b7ec1e80cb4b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36643 } }
I20260806 22:41:50.903996 15989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:50.904019 15989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abec542746046609776b7ec1e80cb4b, State: Initialized, Role: FOLLOWER
I20260806 22:41:50.904168 15989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abec542746046609776b7ec1e80cb4b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36643 } }
I20260806 22:41:50.904261 15989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:50.904309 15989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:50.904383 15989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:50.905023 15989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abec542746046609776b7ec1e80cb4b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36643 } }
I20260806 22:41:50.905151 15989 leader_election.cc:302] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8abec542746046609776b7ec1e80cb4b; no voters:
I20260806 22:41:50.905361 15989 leader_election.cc:288] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:50.905398 15996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:50.905519 15996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [term 1 LEADER]: Becoming Leader. State: Replica: 8abec542746046609776b7ec1e80cb4b, State: Running, Role: LEADER
I20260806 22:41:50.905700 15989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:50.905669 15996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abec542746046609776b7ec1e80cb4b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36643 } }
I20260806 22:41:50.906139 15996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8abec542746046609776b7ec1e80cb4b. Latest consensus state: current_term: 1 leader_uuid: "8abec542746046609776b7ec1e80cb4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abec542746046609776b7ec1e80cb4b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36643 } } }
I20260806 22:41:50.906343 15996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:50.906275 15998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8abec542746046609776b7ec1e80cb4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abec542746046609776b7ec1e80cb4b" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 36643 } } }
I20260806 22:41:50.906603 15998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:50.906919 16011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:50.907482 16011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:50.907807 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:50.901835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58209 (local address 127.16.235.126:36643)
0806 22:41:50.902073 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:50.902082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:50.902124 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:41:50.902632 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:50.902640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:50.902756 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:50.902905 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:50.903141 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:50.904142 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:50.905036 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:50.905668 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:50.905809 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:50.906822 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:50.906826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:50.906826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:50.907056 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:50.907096 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:50.907113 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:50.907214 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:50.907504 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 22:41:50.907537 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":110,"threads_started":1}
I20260806 22:41:50.909022 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:50.910207 16011 catalog_manager.cc:1347] Generated new cluster ID: 892c0f44dd6542b38e299252f72a8e3c
I20260806 22:41:50.910305 16011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:50.914295 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:50.910559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37859 (local address 127.16.235.126:36643)
0806 22:41:50.910623 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:50.910625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:50.910882 (+ 257us) server_negotiation.cc:407] Connection header received
0806 22:41:50.910931 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:50.910932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:50.910944 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:50.911016 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:50.911133 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:50.911877 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:50.912863 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:50.913188 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:50.913267 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:50.914014 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:50.914016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:50.914016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:50.914021 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:50.914037 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:50.914042 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:50.914103 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:50.914198 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:41:50.914210 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:41:50.940227 16011 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:50.940850 16011 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:50.946113 16011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8abec542746046609776b7ec1e80cb4b: Generated new TSK 0
I20260806 22:41:50.946362 16011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:50.951411 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 15902
W20260806 22:41:50.966642 15881 connection.cc:535] client connection to 127.16.235.126:36643 recv error: Network error: failed to read from TLS socket (remote: 127.16.235.126:36643): Connection reset by peer (error 104)
2026-08-06T22:41:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:51 dist-test-slave-3kk6 krb5kdc[16034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:51 dist-test-slave-3kk6 krb5kdc[16034](info): set up 2 sockets
Aug 06 22:41:51 dist-test-slave-3kk6 krb5kdc[16034](info): commencing operation
krb5kdc: starting...
W20260806 22:41:52.189894 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:52 dist-test-slave-3kk6 krb5kdc[16034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:52.321774 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40763
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:45059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:40763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:52.409323 16049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:52.409672 16049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:52.409735 16049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:52.411428 16049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:52.411494 16049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:52.411509 16049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:52.411522 16049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:52.411535 16049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:52.413165 16049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:45059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:40763
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:52.413410 16049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:52.413602 16049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:52.418900 16056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:52.418911 16059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:52.418911 16057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:52.419202 16049 server_base.cc:1034] running on GCE node
I20260806 22:41:52.419528 16049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:52.419801 16049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:52.420964 16049 hybrid_clock.cc:648] HybridClock initialized: now 1786056112420950 us; error 33 us; skew 500 ppm
Aug 06 22:41:52 dist-test-slave-3kk6 krb5kdc[16034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056112, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:52.423772 16049 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:52.424305 16049 webserver.cc:460] Webserver started at http://127.16.235.126:40999/ using document root <none> and password file <none>
I20260806 22:41:52.424517 16049 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:52.424561 16049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:52.424655 16049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:52.425817 16049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "37aa354a570d4fa6af52819cbff73bfa"
format_stamp: "Formatted at 2026-08-06 22:41:52 on dist-test-slave-3kk6"
server_key: "82465d9fb3ca2358cd0038ef873d0e95"
server_key_iv: "7f6b554ab911162e3dd29f2bf62264a2"
server_key_version: "encryptionkey@0"
I20260806 22:41:52.426139 16049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "37aa354a570d4fa6af52819cbff73bfa"
format_stamp: "Formatted at 2026-08-06 22:41:52 on dist-test-slave-3kk6"
server_key: "82465d9fb3ca2358cd0038ef873d0e95"
server_key_iv: "7f6b554ab911162e3dd29f2bf62264a2"
server_key_version: "encryptionkey@0"
I20260806 22:41:52.427655 16049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:41:52.428439 16068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:52.428622 16049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:52.428722 16049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "37aa354a570d4fa6af52819cbff73bfa"
format_stamp: "Formatted at 2026-08-06 22:41:52 on dist-test-slave-3kk6"
server_key: "82465d9fb3ca2358cd0038ef873d0e95"
server_key_iv: "7f6b554ab911162e3dd29f2bf62264a2"
server_key_version: "encryptionkey@0"
I20260806 22:41:52.428784 16049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:52.432245 16049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:52.434506 16049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:52.434633 16049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:52.439069 16049 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:40763
I20260806 22:41:52.439141 16136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:40763 every 8 connection(s)
I20260806 22:41:52.439502 16049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:52.440616 16137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:52.443240 16137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa: Bootstrap starting.
I20260806 22:41:52.443888 16137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:52.444206 16137 log.cc:824] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:52.445026 16137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa: No bootstrap required, opened a new log
I20260806 22:41:52.446755 16137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37aa354a570d4fa6af52819cbff73bfa" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40763 } }
I20260806 22:41:52.446929 16137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:52.446983 16137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37aa354a570d4fa6af52819cbff73bfa, State: Initialized, Role: FOLLOWER
I20260806 22:41:52.447166 16137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37aa354a570d4fa6af52819cbff73bfa" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40763 } }
I20260806 22:41:52.447275 16137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:52.447329 16137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:52.447386 16137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:52.448329 16137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37aa354a570d4fa6af52819cbff73bfa" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40763 } }
I20260806 22:41:52.448464 16137 leader_election.cc:302] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37aa354a570d4fa6af52819cbff73bfa; no voters:
I20260806 22:41:52.448449 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 16049
I20260806 22:41:52.448632 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:52.448711 16137 leader_election.cc:288] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:52.448802 16140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:52.448859 17325 external_mini_cluster.cc:1428] Setting key a86c77b599e00972e72a12c5ad1724bf
I20260806 22:41:52.449007 16140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [term 1 LEADER]: Becoming Leader. State: Replica: 37aa354a570d4fa6af52819cbff73bfa, State: Running, Role: LEADER
I20260806 22:41:52.449033 16137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:52.449117 16140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37aa354a570d4fa6af52819cbff73bfa" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40763 } }
I20260806 22:41:52.449523 16140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37aa354a570d4fa6af52819cbff73bfa. Latest consensus state: current_term: 1 leader_uuid: "37aa354a570d4fa6af52819cbff73bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37aa354a570d4fa6af52819cbff73bfa" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40763 } } }
I20260806 22:41:52.449618 16140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:52.449538 16142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37aa354a570d4fa6af52819cbff73bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37aa354a570d4fa6af52819cbff73bfa" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40763 } } }
I20260806 22:41:52.449863 16142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:52.450618 16158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:52.451301 16158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:52.451923 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:52.449622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37527 (local address 127.16.235.126:40763)
0806 22:41:52.449910 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:52.449919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:52.450104 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:41:52.450251 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:52.450259 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:52.450292 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:52.450781 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:52.451439 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:52.451442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:52.451442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:52.451447 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:52.451482 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:52.451497 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:52.451568 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:52.451694 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 22:41:52.451739 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":109,"threads_started":1}
I20260806 22:41:52.452874 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:52.453169 16158 catalog_manager.cc:1347] Generated new cluster ID: ff2f1bcfea5d4b2c9eb9a947e555360c
I20260806 22:41:52.453223 16158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:52.455710 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:52.454578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41709 (local address 127.16.235.126:40763)
0806 22:41:52.454617 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:52.454618 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:52.454628 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:41:52.454724 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:52.454725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:52.454746 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:52.454816 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:52.455482 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:52.455484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:52.455484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:52.455488 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:52.455506 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:52.455510 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:52.455574 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:52.455628 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:41:52.455642 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:41:52.457983 16158 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:52.458148 16158 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:52.463120 16158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37aa354a570d4fa6af52819cbff73bfa: Generated new TSK 0
I20260806 22:41:52.463342 16158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:52.468355 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 16049
2026-08-06T22:41:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:52 dist-test-slave-3kk6 krb5kdc[16190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:52 dist-test-slave-3kk6 krb5kdc[16190](info): set up 2 sockets
Aug 06 22:41:52 dist-test-slave-3kk6 krb5kdc[16190](info): commencing operation
krb5kdc: starting...
W20260806 22:41:53.685490 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:53 dist-test-slave-3kk6 krb5kdc[16190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:53.813829 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:39995
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:39995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:53.899562 16209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:53.899807 16209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:53.899849 16209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:53.901008 16209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:53.901078 16209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:53.901103 16209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:53.901124 16209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:53.901153 16209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:53.902693 16209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:39995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:39995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:53.902979 16209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:53.903218 16209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:53.908206 16216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:53.908214 16219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:53.908430 16209 server_base.cc:1034] running on GCE node
W20260806 22:41:53.908262 16217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:53.908777 16209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:53.909145 16209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:53.910307 16209 hybrid_clock.cc:648] HybridClock initialized: now 1786056113910290 us; error 46 us; skew 500 ppm
Aug 06 22:41:53 dist-test-slave-3kk6 krb5kdc[16190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056113, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:53.913098 16209 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:53.913552 16209 webserver.cc:460] Webserver started at http://127.16.235.126:40577/ using document root <none> and password file <none>
I20260806 22:41:53.913797 16209 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:53.913844 16209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:53.913995 16209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:53.915186 16209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "eba7f3a75ddc429aa2dcc0daf4106a84"
format_stamp: "Formatted at 2026-08-06 22:41:53 on dist-test-slave-3kk6"
server_key: "8fd09c0a679a422eb0a3507530c839ff"
server_key_iv: "e6642f5070c0a5d3cbcb909cc9de071b"
server_key_version: "encryptionkey@0"
I20260806 22:41:53.915560 16209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "eba7f3a75ddc429aa2dcc0daf4106a84"
format_stamp: "Formatted at 2026-08-06 22:41:53 on dist-test-slave-3kk6"
server_key: "8fd09c0a679a422eb0a3507530c839ff"
server_key_iv: "e6642f5070c0a5d3cbcb909cc9de071b"
server_key_version: "encryptionkey@0"
I20260806 22:41:53.916920 16209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:41:53.917623 16228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:53.917814 16209 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:41:53.917927 16209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "eba7f3a75ddc429aa2dcc0daf4106a84"
format_stamp: "Formatted at 2026-08-06 22:41:53 on dist-test-slave-3kk6"
server_key: "8fd09c0a679a422eb0a3507530c839ff"
server_key_iv: "e6642f5070c0a5d3cbcb909cc9de071b"
server_key_version: "encryptionkey@0"
I20260806 22:41:53.918001 16209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:53.932548 16209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:53.935106 16209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:53.935298 16209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:53.939460 16209 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:39995
I20260806 22:41:53.939518 16297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:39995 every 8 connection(s)
I20260806 22:41:53.939961 16209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:53.940858 16298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:53.943339 16298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84: Bootstrap starting.
I20260806 22:41:53.943959 16298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:53.944252 16298 log.cc:824] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:53.944983 16298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84: No bootstrap required, opened a new log
I20260806 22:41:53.946584 16298 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7f3a75ddc429aa2dcc0daf4106a84" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 39995 } }
I20260806 22:41:53.946797 16298 raft_consensus.cc:374] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:53.946859 16298 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba7f3a75ddc429aa2dcc0daf4106a84, State: Initialized, Role: FOLLOWER
I20260806 22:41:53.947042 16298 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7f3a75ddc429aa2dcc0daf4106a84" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 39995 } }
I20260806 22:41:53.947131 16298 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:53.947183 16298 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:53.947237 16298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:53.948145 16298 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7f3a75ddc429aa2dcc0daf4106a84" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 39995 } }
I20260806 22:41:53.948264 16298 leader_election.cc:302] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba7f3a75ddc429aa2dcc0daf4106a84; no voters:
I20260806 22:41:53.948454 16298 leader_election.cc:288] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:53.948488 16301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:53.948673 16301 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [term 1 LEADER]: Becoming Leader. State: Replica: eba7f3a75ddc429aa2dcc0daf4106a84, State: Running, Role: LEADER
I20260806 22:41:53.948729 16298 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:53.948794 16301 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7f3a75ddc429aa2dcc0daf4106a84" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 39995 } }
I20260806 22:41:53.949225 16303 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba7f3a75ddc429aa2dcc0daf4106a84. Latest consensus state: current_term: 1 leader_uuid: "eba7f3a75ddc429aa2dcc0daf4106a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7f3a75ddc429aa2dcc0daf4106a84" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 39995 } } }
I20260806 22:41:53.949332 16303 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:53.949222 16302 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eba7f3a75ddc429aa2dcc0daf4106a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba7f3a75ddc429aa2dcc0daf4106a84" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 39995 } } }
I20260806 22:41:53.949463 16302 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:53.949779 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 16209
I20260806 22:41:53.949903 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:53.950132 17325 external_mini_cluster.cc:1428] Setting key a5fab6204db068049a897a5f1ae213d5
I20260806 22:41:53.950758 16314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:53.951290 16314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:53.953922 16314 catalog_manager.cc:1347] Generated new cluster ID: 7e0175372f874704a0a8da9c6768ad0f
I20260806 22:41:53.953979 16314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:41:53 dist-test-slave-3kk6 krb5kdc[16190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:53.961637 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:53.950719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51929 (local address 127.16.235.126:39995)
0806 22:41:53.950916 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:53.950926 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:53.950965 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:41:53.951110 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:53.951116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:53.951159 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:53.951293 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:53.952032 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:53.953428 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:53.954403 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:53.954903 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:53.955039 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:53.957504 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:53.957508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:53.957509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:53.957522 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:53.959974 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:53.960386 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:53.960390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:53.960390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:53.960501 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:53.960722 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:53.960723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:53.960724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:53.960993 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:53.961174 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:53.961278 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:41:53.961444 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260806 22:41:53.962612 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:53.968364 16314 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:53.969079 16314 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:53.975214 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:53.964071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33479 (local address 127.16.235.126:39995)
0806 22:41:53.964110 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:53.964112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:53.968047 (+ 3935us) server_negotiation.cc:407] Connection header received
0806 22:41:53.968105 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:53.968106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:53.968120 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:53.968187 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:53.969677 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:53.970167 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:53.970951 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:53.971157 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:53.971233 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:53.972513 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:53.972514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:53.972515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:53.972522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:53.973968 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:53.974333 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:53.974334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:53.974334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:53.974400 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:53.974611 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:53.974612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:53.974612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:53.974777 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:53.974892 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:53.975004 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 22:41:53.975096 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:41:53.978204 16314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba7f3a75ddc429aa2dcc0daf4106a84: Generated new TSK 0
I20260806 22:41:53.978418 16314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:53.983289 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 16209
2026-08-06T22:41:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:54 dist-test-slave-3kk6 krb5kdc[16340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:54 dist-test-slave-3kk6 krb5kdc[16340](info): set up 2 sockets
Aug 06 22:41:54 dist-test-slave-3kk6 krb5kdc[16340](info): commencing operation
krb5kdc: starting...
W20260806 22:41:55.207854 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:55 dist-test-slave-3kk6 krb5kdc[16340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:55.351966 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38517
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:38517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:55.438514 16363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:55.438779 16363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:55.438807 16363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:55.439922 16363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:55.439966 16363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:55.439982 16363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:55.439994 16363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:55.440006 16363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:55.441548 16363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:38517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:38517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:55.441792 16363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:55.441980 16363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:55.447201 16369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:55.447190 16373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:55.447170 16370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:55.447440 16363 server_base.cc:1034] running on GCE node
I20260806 22:41:55.447695 16363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:55.448012 16363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:55.449193 16363 hybrid_clock.cc:648] HybridClock initialized: now 1786056115449177 us; error 50 us; skew 500 ppm
Aug 06 22:41:55 dist-test-slave-3kk6 krb5kdc[16340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056115, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:55.451844 16363 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:55.452251 16363 webserver.cc:460] Webserver started at http://127.16.235.126:35161/ using document root <none> and password file <none>
I20260806 22:41:55.452461 16363 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:55.452507 16363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:55.452603 16363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:55.453656 16363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "297ffd8ad7ca416d915f526dc8a7dce9"
format_stamp: "Formatted at 2026-08-06 22:41:55 on dist-test-slave-3kk6"
server_key: "fbed82d9dc5038122765ab73ad844ce0"
server_key_iv: "68ca7cea5ca94b0ba79417d6a3a1fac4"
server_key_version: "encryptionkey@0"
I20260806 22:41:55.453969 16363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "297ffd8ad7ca416d915f526dc8a7dce9"
format_stamp: "Formatted at 2026-08-06 22:41:55 on dist-test-slave-3kk6"
server_key: "fbed82d9dc5038122765ab73ad844ce0"
server_key_iv: "68ca7cea5ca94b0ba79417d6a3a1fac4"
server_key_version: "encryptionkey@0"
I20260806 22:41:55.455451 16363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:55.456187 16384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:55.456387 16363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:55.456470 16363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "297ffd8ad7ca416d915f526dc8a7dce9"
format_stamp: "Formatted at 2026-08-06 22:41:55 on dist-test-slave-3kk6"
server_key: "fbed82d9dc5038122765ab73ad844ce0"
server_key_iv: "68ca7cea5ca94b0ba79417d6a3a1fac4"
server_key_version: "encryptionkey@0"
I20260806 22:41:55.456533 16363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:55.479306 16363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:55.481580 16363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:55.481742 16363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:55.486363 16363 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:38517
I20260806 22:41:55.486408 16450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:38517 every 8 connection(s)
I20260806 22:41:55.486876 16363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:55.488000 16452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:55.488464 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 16363
I20260806 22:41:55.488562 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:55.488767 17325 external_mini_cluster.cc:1428] Setting key d1c7a8f3f67a12380d4f815987ae66ca
I20260806 22:41:55.491545 16452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9: Bootstrap starting.
I20260806 22:41:55.492334 16452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:55.492660 16452 log.cc:824] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:55.493427 16452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9: No bootstrap required, opened a new log
I20260806 22:41:55.495330 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:55.489423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47283 (local address 127.16.235.126:38517)
0806 22:41:55.489645 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:55.489655 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:55.489694 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:41:55.490263 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:55.490273 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:55.490337 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:55.490503 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:55.490793 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:55.492218 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:55.493181 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:55.493660 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:55.493806 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:55.494634 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:55.494639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:55.494639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:55.494876 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:55.494908 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:55.494920 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:55.494999 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:55.495076 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:41:55.495144 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260806 22:41:55.495564 16452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297ffd8ad7ca416d915f526dc8a7dce9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38517 } }
I20260806 22:41:55.495766 16452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:55.495806 16452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297ffd8ad7ca416d915f526dc8a7dce9, State: Initialized, Role: FOLLOWER
I20260806 22:41:55.495936 16452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297ffd8ad7ca416d915f526dc8a7dce9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38517 } }
I20260806 22:41:55.496035 16452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:55.496080 16452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:55.496132 16452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:55.496447 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:55.496996 16452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297ffd8ad7ca416d915f526dc8a7dce9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38517 } }
I20260806 22:41:55.497145 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 297ffd8ad7ca416d915f526dc8a7dce9; no voters:
I20260806 22:41:55.497354 16452 leader_election.cc:288] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:55.497433 16459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:55.497670 16459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [term 1 LEADER]: Becoming Leader. State: Replica: 297ffd8ad7ca416d915f526dc8a7dce9, State: Running, Role: LEADER
I20260806 22:41:55.497709 16452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:55.497851 16459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297ffd8ad7ca416d915f526dc8a7dce9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38517 } }
I20260806 22:41:55.498272 16465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 297ffd8ad7ca416d915f526dc8a7dce9. Latest consensus state: current_term: 1 leader_uuid: "297ffd8ad7ca416d915f526dc8a7dce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297ffd8ad7ca416d915f526dc8a7dce9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38517 } } }
I20260806 22:41:55.498272 16460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "297ffd8ad7ca416d915f526dc8a7dce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297ffd8ad7ca416d915f526dc8a7dce9" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 38517 } } }
I20260806 22:41:55.498389 16465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:55.498394 16460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:55.498728 16475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:55.499536 16475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:55.501304 16475 catalog_manager.cc:1347] Generated new cluster ID: 47c557e529da4ca390e68550e90931f6
I20260806 22:41:55.501363 16475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:55.502713 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:55.499064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42549 (local address 127.16.235.126:38517)
0806 22:41:55.499104 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:55.499107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:55.499481 (+ 374us) server_negotiation.cc:407] Connection header received
0806 22:41:55.499594 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:55.499595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:55.499605 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:55.499656 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:55.499839 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:55.500406 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:55.501270 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:55.501567 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:55.501605 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:55.502449 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:55.502451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:55.502451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:55.502456 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:55.502471 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:55.502475 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:55.502525 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:55.502571 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 22:41:55.502597 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:41:55.511718 16475 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:55.512480 16475 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:55.517429 16475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 297ffd8ad7ca416d915f526dc8a7dce9: Generated new TSK 0
I20260806 22:41:55.517675 16475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:55.521850 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 16363
2026-08-06T22:41:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:55 dist-test-slave-3kk6 krb5kdc[16492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:55 dist-test-slave-3kk6 krb5kdc[16492](info): set up 2 sockets
Aug 06 22:41:55 dist-test-slave-3kk6 krb5kdc[16492](info): commencing operation
krb5kdc: starting...
W20260806 22:41:56.767387 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:56 dist-test-slave-3kk6 krb5kdc[16492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:56.899384 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34239
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:34239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:56.986081 16510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:56.986317 16510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:56.986342 16510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:56.987655 16510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:56.987715 16510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:56.987730 16510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:56.987743 16510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:56.987756 16510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:56.989346 16510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:34239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:34239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:56.989656 16510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:56.989924 16510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:56.995404 16516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:56.995397 16517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:56.995472 16510 server_base.cc:1034] running on GCE node
W20260806 22:41:56.995402 16520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:56.996021 16510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:56.996381 16510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:56.997553 16510 hybrid_clock.cc:648] HybridClock initialized: now 1786056116997526 us; error 54 us; skew 500 ppm
Aug 06 22:41:56 dist-test-slave-3kk6 krb5kdc[16492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056116, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:57.004436 16510 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:57.004969 16510 webserver.cc:460] Webserver started at http://127.16.235.126:35879/ using document root <none> and password file <none>
I20260806 22:41:57.005244 16510 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:57.005324 16510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:57.005493 16510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:57.006732 16510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "2a1d3b5415204d338e06c8e9ccfff0d5"
format_stamp: "Formatted at 2026-08-06 22:41:56 on dist-test-slave-3kk6"
server_key: "5ecbfbc7d0f4cd70507c4b44bbda04ab"
server_key_iv: "b0381fe53fa10ca7f40496db3ad70e01"
server_key_version: "encryptionkey@0"
I20260806 22:41:57.007146 16510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "2a1d3b5415204d338e06c8e9ccfff0d5"
format_stamp: "Formatted at 2026-08-06 22:41:56 on dist-test-slave-3kk6"
server_key: "5ecbfbc7d0f4cd70507c4b44bbda04ab"
server_key_iv: "b0381fe53fa10ca7f40496db3ad70e01"
server_key_version: "encryptionkey@0"
I20260806 22:41:57.008646 16510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:57.009536 16532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:57.009778 16510 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:57.009920 16510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "2a1d3b5415204d338e06c8e9ccfff0d5"
format_stamp: "Formatted at 2026-08-06 22:41:56 on dist-test-slave-3kk6"
server_key: "5ecbfbc7d0f4cd70507c4b44bbda04ab"
server_key_iv: "b0381fe53fa10ca7f40496db3ad70e01"
server_key_version: "encryptionkey@0"
I20260806 22:41:57.010010 16510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:57.013639 16510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:57.015965 16510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:57.016144 16510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:57.020377 16510 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:34239
I20260806 22:41:57.020452 16599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:34239 every 8 connection(s)
I20260806 22:41:57.020794 16510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:57.021931 16600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:57.024495 16600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5: Bootstrap starting.
I20260806 22:41:57.025075 16600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:57.025328 16600 log.cc:824] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:57.025744 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 16510
I20260806 22:41:57.025899 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:57.026127 17325 external_mini_cluster.cc:1428] Setting key 74e1d1edfadee75a7a56616e91f02e81
I20260806 22:41:57.026167 16600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5: No bootstrap required, opened a new log
I20260806 22:41:57.028702 16600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1d3b5415204d338e06c8e9ccfff0d5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34239 } }
I20260806 22:41:57.028925 16600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:41:57.029003 16600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a1d3b5415204d338e06c8e9ccfff0d5, State: Initialized, Role: FOLLOWER
I20260806 22:41:57.029139 16600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1d3b5415204d338e06c8e9ccfff0d5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34239 } }
I20260806 22:41:57.029251 16600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:41:57.029258 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:57.026744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50515 (local address 127.16.235.126:34239)
0806 22:41:57.026969 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:57.026977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:57.027001 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:41:57.027613 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:57.027621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:57.027650 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:57.027779 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:57.028542 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:57.028545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:57.028545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:57.028549 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:57.028587 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:57.028597 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:57.028653 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:57.028755 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 22:41:57.028842 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":114,"threads_started":1}
I20260806 22:41:57.029326 16600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:41:57.029378 16600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:57.030265 16600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1d3b5415204d338e06c8e9ccfff0d5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34239 } }
I20260806 22:41:57.030323 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:57.030395 16600 leader_election.cc:302] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a1d3b5415204d338e06c8e9ccfff0d5; no voters:
I20260806 22:41:57.030563 16600 leader_election.cc:288] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:41:57.030609 16607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:57.030812 16607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [term 1 LEADER]: Becoming Leader. State: Replica: 2a1d3b5415204d338e06c8e9ccfff0d5, State: Running, Role: LEADER
I20260806 22:41:57.030848 16600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:57.030968 16607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1d3b5415204d338e06c8e9ccfff0d5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34239 } }
I20260806 22:41:57.031365 16607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a1d3b5415204d338e06c8e9ccfff0d5. Latest consensus state: current_term: 1 leader_uuid: "2a1d3b5415204d338e06c8e9ccfff0d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1d3b5415204d338e06c8e9ccfff0d5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34239 } } }
I20260806 22:41:57.031644 16607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:57.031512 16611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a1d3b5415204d338e06c8e9ccfff0d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a1d3b5415204d338e06c8e9ccfff0d5" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 34239 } } }
I20260806 22:41:57.032105 16611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:57.032351 16623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:57.033320 16623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:57.033422 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:57.031995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49653 (local address 127.16.235.126:34239)
0806 22:41:57.032038 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:57.032040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:57.032308 (+ 268us) server_negotiation.cc:407] Connection header received
0806 22:41:57.032414 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:57.032415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:57.032421 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:57.032452 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:57.033175 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:57.033176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:57.033177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:41:57.033181 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:57.033195 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:41:57.033200 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:57.033230 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:57.033341 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:41:57.033356 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:41:57.035264 16623 catalog_manager.cc:1347] Generated new cluster ID: 40b713d22abb442bae699b1b2c185e6e
I20260806 22:41:57.035329 16623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:57.065963 16623 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:57.066175 16623 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:57.081838 16623 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a1d3b5415204d338e06c8e9ccfff0d5: Generated new TSK 0
I20260806 22:41:57.082116 16623 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:41:57.089838 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 16510
2026-08-06T22:41:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1562 ms)
[----------] 18 tests from AuthTokenIssuingTest (27935 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:41:57 dist-test-slave-3kk6 krb5kdc[16647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:41:57 dist-test-slave-3kk6 krb5kdc[16647](info): set up 2 sockets
Aug 06 22:41:57 dist-test-slave-3kk6 krb5kdc[16647](info): commencing operation
krb5kdc: starting...
W20260806 22:41:58.320204 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:41:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:41:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:41:58.453081 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40739
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:40739,127.16.235.125:36009,127.16.235.124:41797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:41:58.537955 16667 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:41:58.538220 16667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:58.538239 16667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:58.538261 16667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:58.539737 16667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:58.539813 16667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:58.539829 16667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:58.539844 16667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:58.539856 16667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:58.541568 16667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:40739,127.16.235.125:36009,127.16.235.124:41797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:40739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:58.541812 16667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:58.542018 16667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:58.547298 16675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:58.547442 16667 server_base.cc:1034] running on GCE node
W20260806 22:41:58.547305 16676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:58.547355 16678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:58.547907 16667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:58.548254 16667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:58.549432 16667 hybrid_clock.cc:648] HybridClock initialized: now 1786056118549428 us; error 41 us; skew 500 ppm
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:58.552280 16667 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:41:58.552772 16667 webserver.cc:460] Webserver started at http://127.16.235.126:40135/ using document root <none> and password file <none>
I20260806 22:41:58.553018 16667 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:58.553092 16667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:58.553246 16667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:58.554428 16667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "92fd2db4e1e04924b5fe070bc92ac581"
format_stamp: "Formatted at 2026-08-06 22:41:58 on dist-test-slave-3kk6"
server_key: "c9555281705d028a5fc9eabe702872d9"
server_key_iv: "011be3f83f16347f950f0f257a807b13"
server_key_version: "encryptionkey@0"
I20260806 22:41:58.554852 16667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "92fd2db4e1e04924b5fe070bc92ac581"
format_stamp: "Formatted at 2026-08-06 22:41:58 on dist-test-slave-3kk6"
server_key: "c9555281705d028a5fc9eabe702872d9"
server_key_iv: "011be3f83f16347f950f0f257a807b13"
server_key_version: "encryptionkey@0"
I20260806 22:41:58.556306 16667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:58.557144 16689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:58.557330 16667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:58.557451 16667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "92fd2db4e1e04924b5fe070bc92ac581"
format_stamp: "Formatted at 2026-08-06 22:41:58 on dist-test-slave-3kk6"
server_key: "c9555281705d028a5fc9eabe702872d9"
server_key_iv: "011be3f83f16347f950f0f257a807b13"
server_key_version: "encryptionkey@0"
I20260806 22:41:58.557531 16667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:58.576354 16667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:58.579137 16667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:58.579332 16667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:58.583750 16667 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:40739
I20260806 22:41:58.583863 16764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:40739 every 8 connection(s)
I20260806 22:41:58.584224 16667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:41:58.585196 16767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:58.586997 16767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:41:58.589004 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 16667
I20260806 22:41:58.589162 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:41:58.589370 17325 external_mini_cluster.cc:1428] Setting key e37f78ab5a7728a075e3c0945a0258f3
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.235.125@KRBTEST.COM; defaulting to no policy
I20260806 22:41:58.598802 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.587770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40739 (local address 127.0.0.1:58862)
0806 22:41:58.587965 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.588001 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:58.588507 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:58.588773 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:58.588775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:58.589066 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:58.589380 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.589385 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.590958 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.590959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.592036 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.592040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.592551 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.592552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.592640 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.593240 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:58.593251 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:58.595186 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:58.597136 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.597143 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.597146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.597380 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.597637 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.597638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.597639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.597729 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.598121 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:58.598126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:58.598190 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 22:41:58.598301 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:41:58.598582 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":119,"thread_start_us":102,"threads_started":1}
I20260806 22:41:58.598810 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.587801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.16.235.126:40739)
0806 22:41:58.587994 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.588011 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.588077 (+ 66us) server_negotiation.cc:407] Connection header received
0806 22:41:58.588644 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.588646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.588663 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.588743 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.589512 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.590843 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.592137 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.592457 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.592543 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.595303 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.595310 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.595310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.595318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.597036 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.597470 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.597474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.597474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.597565 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.597798 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.597799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.597799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.598046 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.598143 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.598336 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.598514 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260806 22:41:58.599694 16767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:41:58.600153 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.600034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:36009 (local address 127.0.0.1:41292)
0806 22:41:58.600081 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.600128 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:41:58.600303 16695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
I20260806 22:41:58.600729 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.600627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:36009 (local address 127.0.0.1:41294)
0806 22:41:58.600678 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.600705 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:41:58.600948 16767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.125:36009: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab.
Entry for principal kudu/127.16.235.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab.
I20260806 22:41:58.617938 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.125:36009
--webserver_interface=127.16.235.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab
--principal=kudu/127.16.235.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:40739,127.16.235.125:36009,127.16.235.124:41797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:41:58.650295 16767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } attempt: 1
I20260806 22:41:58.650971 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.650816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:36009 (local address 127.0.0.1:41302)
0806 22:41:58.650889 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.650939 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 22:41:58.651361 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.651257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:36009 (local address 127.0.0.1:41318)
0806 22:41:58.651307 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.651340 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:41:58.651454 16767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.125:36009: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
W20260806 22:41:58.706112 16775 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:41:58.706387 16775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:58.706422 16775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:58.706470 16775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:58.707754 16775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:58.707829 16775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:58.707851 16775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:58.707880 16775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:58.707911 16775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:58.709458 16775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:40739,127.16.235.125:36009,127.16.235.124:41797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab
--principal=kudu/127.16.235.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.125:36009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.235.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:58.709767 16775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:58.710028 16775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:58.715279 16785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:58.715278 16780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:58.715318 16783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:58.715720 16775 server_base.cc:1034] running on GCE node
I20260806 22:41:58.715992 16775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:58.716372 16775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:58.717545 16775 hybrid_clock.cc:648] HybridClock initialized: now 1786056118717527 us; error 46 us; skew 500 ppm
I20260806 22:41:58.719595 16767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } attempt: 2
I20260806 22:41:58.720033 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.719886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:36009 (local address 127.0.0.1:41324)
0806 22:41:58.719961 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.719999 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:58.720429 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.720354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:36009 (local address 127.0.0.1:41340)
0806 22:41:58.720390 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.720413 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 22:41:58.720537 16767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.125:36009: Network error: Client connection negotiation failed: client connection to 127.16.235.125:36009: connect: Connection refused (error 111)
I20260806 22:41:58.720666 16775 init.cc:377] Logged in from keytab as kudu/127.16.235.125@KRBTEST.COM (short username kudu)
I20260806 22:41:58.721109 16775 webserver.cc:460] Webserver started at http://127.16.235.125:40427/ using document root <none> and password file <none>
I20260806 22:41:58.721313 16775 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:58.721359 16775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:58.721464 16775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:58.722558 16775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/data/instance:
uuid: "892201bf235045218b84ef06c422d021"
format_stamp: "Formatted at 2026-08-06 22:41:58 on dist-test-slave-3kk6"
server_key: "98f23f0d940509bfda78ab57c13156af"
server_key_iv: "f81a9333c225f8f8e8138401a575db2a"
server_key_version: "encryptionkey@0"
I20260806 22:41:58.722980 16775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/wal/instance:
uuid: "892201bf235045218b84ef06c422d021"
format_stamp: "Formatted at 2026-08-06 22:41:58 on dist-test-slave-3kk6"
server_key: "98f23f0d940509bfda78ab57c13156af"
server_key_iv: "f81a9333c225f8f8e8138401a575db2a"
server_key_version: "encryptionkey@0"
I20260806 22:41:58.724479 16775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:58.725255 16799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:58.725440 16775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:41:58.725541 16775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/wal
uuid: "892201bf235045218b84ef06c422d021"
format_stamp: "Formatted at 2026-08-06 22:41:58 on dist-test-slave-3kk6"
server_key: "98f23f0d940509bfda78ab57c13156af"
server_key_iv: "f81a9333c225f8f8e8138401a575db2a"
server_key_version: "encryptionkey@0"
I20260806 22:41:58.725605 16775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:58.735817 16775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:58.738180 16775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:58.738325 16775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:58.743686 16775 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.125:36009
I20260806 22:41:58.743768 16874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.125:36009 every 8 connection(s)
I20260806 22:41:58.744128 16775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
I20260806 22:41:58.745067 16875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:58.746770 16875 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:58.754395 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 16775
I20260806 22:41:58.754547 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-1/wal/instance
I20260806 22:41:58.754791 17325 external_mini_cluster.cc:1428] Setting key b2d81527be2f2395f052817deb1b7c85
I20260806 22:41:58.756644 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.747562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.16.235.126:40739)
0806 22:41:58.747628 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.747630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.747863 (+ 233us) server_negotiation.cc:407] Connection header received
0806 22:41:58.748342 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.748343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.748355 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.748422 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.749055 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.749562 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.750593 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.751006 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.751107 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.753569 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.753570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.753571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.753578 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.755112 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.755639 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.755640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.755640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.755699 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.755997 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.755998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.755998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.756110 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.756161 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.756499 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.756576 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:41:58.757031 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.747534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40739 (local address 127.0.0.1:58870)
0806 22:41:58.747724 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.747770 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:58.748251 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:58.748502 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:58.748506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:58.748764 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:58.748978 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.748982 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.749642 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.749643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.750432 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.750435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.751122 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.751124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.751310 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.752054 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:58.752063 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:58.753471 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:58.755197 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.755206 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.755209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.755548 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.755763 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.755764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.755764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.755931 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.756174 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:58.756181 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:58.756360 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 22:41:58.756481 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 22:41:58.756671 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260806 22:41:58.757642 16875 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.235.124@KRBTEST.COM; defaulting to no policy
I20260806 22:41:58.766552 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.757955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:36009 (local address 127.0.0.1:41344)
0806 22:41:58.758011 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.758023 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:58.758070 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:58.758409 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:58.758410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:58.758530 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:58.758582 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.758584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.759953 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.759954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.760628 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.760631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.761377 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.761378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.761432 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.761949 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:58.761953 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:58.763405 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:58.765521 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.765522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.765522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.765690 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.765888 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.765889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.765889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.765941 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.766285 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:58.766285 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:58.766324 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:41:58.766392 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:41:58.766479 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:41:58.766714 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.758009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.16.235.125:36009)
0806 22:41:58.758221 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.758227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.758256 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:41:58.758283 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.758285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.758315 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.758395 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.758695 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.759865 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.760707 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.761283 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.761381 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.763500 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.763511 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.763512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.763520 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.765440 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.765765 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.765769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.765769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.765828 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.765994 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.765995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.765995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.766215 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.766283 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.766420 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.766532 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":175,"thread_start_us":123,"threads_started":1}
I20260806 22:41:58.767365 16875 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.124@KRBTEST.COM" created.
I20260806 22:41:58.767798 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.767670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42172)
0806 22:41:58.767706 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.767756 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 22:41:58.767951 16800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
I20260806 22:41:58.768414 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.768312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42176)
0806 22:41:58.768357 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.768388 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:41:58.768538 16875 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:41797: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab.
Entry for principal kudu/127.16.235.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab.
I20260806 22:41:58.785471 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.124:41797
--webserver_interface=127.16.235.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab
--principal=kudu/127.16.235.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:43141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:40739,127.16.235.125:36009,127.16.235.124:41797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:41:58.811721 16767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } attempt: 3
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
I20260806 22:41:58.817790 16875 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } attempt: 1
I20260806 22:41:58.818267 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.818117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42178)
0806 22:41:58.818168 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.818206 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:41:58.818778 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.818616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42194)
0806 22:41:58.818698 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.818737 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 22:41:58.818902 16875 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:41797: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
I20260806 22:41:58.820493 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.812307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41352 (local address 127.16.235.125:36009)
0806 22:41:58.812387 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.812390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.812409 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:41:58.812523 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.812525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.812540 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.812627 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.813016 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.813602 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.814413 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.814715 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.814818 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.817181 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.817183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.817183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.817191 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.819180 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.819625 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.819626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.819626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.819690 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.819957 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.819958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.819959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.820113 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.820175 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.820353 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.820431 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:41:58.820544 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.812228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:36009 (local address 127.0.0.1:41352)
0806 22:41:58.812322 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.812358 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:58.812428 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:58.812635 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:58.812636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:58.812839 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:58.812917 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.812919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.813700 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.813702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.814310 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.814312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.814816 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.814817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.814915 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.815494 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:58.815499 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:58.817060 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:58.819271 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.819272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.819273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.819534 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.819762 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.819763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.819764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.819838 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.820186 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:58.820187 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:58.820244 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 22:41:58.820320 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:41:58.820439 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 22:41:58.821003 16767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:41:58.821439 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.821313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42208)
0806 22:41:58.821371 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.821410 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:41:58.821792 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.821690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42218)
0806 22:41:58.821743 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.821771 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:41:58.821898 16767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:41797: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
I20260806 22:41:58.853101 16767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } attempt: 1
I20260806 22:41:58.853820 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.853613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42230)
0806 22:41:58.853729 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.853784 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 22:41:58.854266 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.854174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42240)
0806 22:41:58.854221 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.854249 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:41:58.854415 16767 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:41797: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
W20260806 22:41:58.875456 16885 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:41:58.875766 16885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:41:58.875801 16885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:41:58.875849 16885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:41:58.877105 16885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:41:58.877185 16885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:41:58.877209 16885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:41:58.877229 16885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:41:58.877262 16885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:41:58.878903 16885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:43141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:40739,127.16.235.125:36009,127.16.235.124:41797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab
--principal=kudu/127.16.235.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.124:41797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.235.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:41:58.879215 16885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:41:58.879480 16885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:41:58.884987 16892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:58.884951 16893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:41:58.884943 16896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:41:58.885370 16885 server_base.cc:1034] running on GCE node
I20260806 22:41:58.885685 16885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:41:58.886078 16885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:41:58.887037 16875 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } attempt: 2
I20260806 22:41:58.887265 16885 hybrid_clock.cc:648] HybridClock initialized: now 1786056118887233 us; error 40 us; skew 500 ppm
I20260806 22:41:58.887485 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.887361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42248)
0806 22:41:58.887419 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.887454 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:41:58.887846 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.887748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42250)
0806 22:41:58.887797 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.887823 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:41:58.887939 16875 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:41797: Network error: Client connection negotiation failed: client connection to 127.16.235.124:41797: connect: Connection refused (error 111)
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:41:58.890251 16885 init.cc:377] Logged in from keytab as kudu/127.16.235.124@KRBTEST.COM (short username kudu)
I20260806 22:41:58.890921 16885 webserver.cc:460] Webserver started at http://127.16.235.124:37045/ using document root <none> and password file <none>
I20260806 22:41:58.891237 16885 fs_manager.cc:359] Metadata directory not provided
I20260806 22:41:58.891287 16885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:41:58.891453 16885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:41:58.892656 16885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/data/instance:
uuid: "230a87af30124046bf17587ab7b3f726"
format_stamp: "Formatted at 2026-08-06 22:41:58 on dist-test-slave-3kk6"
server_key: "fe08d84b16ca258a124317537577509e"
server_key_iv: "cd6d5aa4a7556887edf5b29778f32c8d"
server_key_version: "encryptionkey@0"
I20260806 22:41:58.893075 16885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/wal/instance:
uuid: "230a87af30124046bf17587ab7b3f726"
format_stamp: "Formatted at 2026-08-06 22:41:58 on dist-test-slave-3kk6"
server_key: "fe08d84b16ca258a124317537577509e"
server_key_iv: "cd6d5aa4a7556887edf5b29778f32c8d"
server_key_version: "encryptionkey@0"
I20260806 22:41:58.894553 16885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:41:58.895525 16909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:41:58.895769 16885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:41:58.895861 16885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/wal
uuid: "230a87af30124046bf17587ab7b3f726"
format_stamp: "Formatted at 2026-08-06 22:41:58 on dist-test-slave-3kk6"
server_key: "fe08d84b16ca258a124317537577509e"
server_key_iv: "cd6d5aa4a7556887edf5b29778f32c8d"
server_key_version: "encryptionkey@0"
I20260806 22:41:58.895983 16885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:41:58.918706 16885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:41:58.920928 16885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:41:58.921103 16885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:41:58.925560 16885 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.124:41797
I20260806 22:41:58.925645 16980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.124:41797 every 8 connection(s)
I20260806 22:41:58.926041 16885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
I20260806 22:41:58.927088 16982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:41:58.928911 16982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:41:58.929571 16767 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } attempt: 2
I20260806 22:41:58.931880 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 16885
I20260806 22:41:58.931995 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786055836003438-17325-0/minicluster-data/master-2/wal/instance
I20260806 22:41:58.932183 17325 external_mini_cluster.cc:1428] Setting key d422f2613ce00fa038693d795f5d7ab4
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
I20260806 22:41:58.940332 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.929943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42262)
0806 22:41:58.929991 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.930005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:58.930070 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:58.930867 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:58.930868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:58.931066 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:58.931107 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.931109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.932684 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.932685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.933226 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.933228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.933858 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.933860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.933919 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.934631 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:58.934637 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:58.936668 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:58.938861 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.938863 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.938864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.939113 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.939471 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.939472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.939472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.939534 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.940010 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:58.940011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:58.940051 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 22:41:58.940100 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:41:58.940215 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:41:58.940421 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.929802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58886 (local address 127.16.235.126:40739)
0806 22:41:58.929881 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.929884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.930172 (+ 288us) server_negotiation.cc:407] Connection header received
0806 22:41:58.930836 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.930837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.930850 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.930928 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.931527 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.932054 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.933228 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.933555 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.933621 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.937127 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.937129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.937130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.937138 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.938921 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.939372 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.939374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.939374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.939455 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.939708 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.939710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.939710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.939906 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.939991 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.940243 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.940312 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:41:58.940641 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.929785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:40739 (local address 127.0.0.1:58886)
0806 22:41:58.930037 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.930073 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:58.930717 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:58.930954 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:58.930958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:58.931303 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:58.931437 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.931442 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.932146 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.932147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.933117 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.933120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.933698 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.933699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.933830 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.934597 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:58.934609 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:58.937016 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:58.939009 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.939016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.939019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.939285 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.939513 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.939515 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.939515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.939633 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.939996 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:58.940005 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:58.940089 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 22:41:58.940194 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:41:58.940330 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":106,"threads_started":1}
I20260806 22:41:58.941094 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.930038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42262 (local address 127.16.235.124:41797)
0806 22:41:58.930207 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.930212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.930245 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:41:58.930723 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.930737 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.930767 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.930859 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.931195 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.932570 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.933309 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.933776 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.933859 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.936760 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.936770 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.936770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.936783 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.938774 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.939206 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.939211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.939211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.939334 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.939598 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.939599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.939600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.939920 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.940583 (+ 663us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.940632 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.940753 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260806 22:41:58.941571 16982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:41:58.943332 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.932574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.16.235.124:41797)
0806 22:41:58.932744 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.932746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.933412 (+ 666us) server_negotiation.cc:407] Connection header received
0806 22:41:58.933467 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.933467 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.933478 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.933551 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.934357 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.934986 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.936067 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.936292 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.936354 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.939554 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.939555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.939556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.939562 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.941997 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.942443 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.942444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.942445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.942505 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.942744 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.942745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.942746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.942865 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.943027 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.943093 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.943175 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":114,"threads_started":1}
I20260806 22:41:58.943871 16767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581: Bootstrap starting.
I20260806 22:41:58.944804 16767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:58.945207 16767 log.cc:824] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:58.946164 16767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581: No bootstrap required, opened a new log
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
I20260806 22:41:58.948180 16767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } }
I20260806 22:41:58.948345 16767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92fd2db4e1e04924b5fe070bc92ac581, State: Initialized, Role: FOLLOWER
I20260806 22:41:58.948447 16767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } }
I20260806 22:41:58.948787 16993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:58.948876 16767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:58.948973 16993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:41:58.950490 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.942063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:36009 (local address 127.0.0.1:41368)
0806 22:41:58.942110 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.942120 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:58.942155 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:58.942309 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:58.942310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:58.942461 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:58.942508 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.942510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.943690 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.943691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.944479 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.944481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.944940 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.944941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.945007 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.945528 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:58.945533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:58.947215 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:58.949313 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.949314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.949315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.949531 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.949766 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.949767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.949768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.949830 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.950130 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:58.950131 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:58.950182 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:41:58.950259 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:41:58.950378 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:41:58.951115 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.942132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41368 (local address 127.16.235.125:36009)
0806 22:41:58.942176 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.942178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.942189 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:41:58.942215 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.942216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.942227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.942294 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.942599 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.943354 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.944562 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.944849 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.944932 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.947335 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.947337 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.947337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.947347 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.949208 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.949624 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.949626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.949627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.949690 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.949901 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.949902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.949902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.950049 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.950138 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.950890 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.951020 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:41:58.951169 17010 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:41:58.951376 17010 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:41:58.951872 16982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:41:58.953982 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.944174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.16.235.126:40739)
0806 22:41:58.944216 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.944218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.944226 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:41:58.944255 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.944255 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.944266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.944320 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.945037 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.945716 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.946557 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.946848 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.946926 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.950527 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.950529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.950530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.950538 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.952326 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.952751 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.952752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.952753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.952817 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.953034 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.953035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.953036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.953189 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.953335 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.953797 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.953881 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:41:58.953088 16667 master_runner.cc:186] Error getting master registration for 127.16.235.125:36009: OK, instance_id { permanent_uuid: "892201bf235045218b84ef06c422d021" instance_seqno: 1786056118742334 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:41:58.956265 16667 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
I20260806 22:41:58.960491 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.952132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42274)
0806 22:41:58.952462 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.952473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:58.952522 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:58.952682 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:58.952683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:58.952850 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:58.952893 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.952894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.953852 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.953854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.954624 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.954626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.955244 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.955245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.955289 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.955807 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:58.955813 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:58.957320 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:58.959215 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.959216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.959216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.959468 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.959705 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.959705 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.959705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.959763 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.960080 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:58.960081 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:58.960123 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 22:41:58.960223 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:41:58.960361 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319}
I20260806 22:41:58.961942 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.952397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.16.235.124:41797)
0806 22:41:58.952439 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.952441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.952509 (+ 68us) server_negotiation.cc:407] Connection header received
0806 22:41:58.952578 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.952579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.952593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.952641 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.952981 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.953731 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.954829 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.955140 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.955198 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.957389 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.957390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.957391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.957398 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.959117 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.959553 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.959554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.959555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.959630 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.959833 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.959834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.959834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.960001 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.960079 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.960267 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.961841 (+ 1574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1473,"server-negotiator.queue_time_us":20}
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
I20260806 22:41:58.964139 16982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726: Bootstrap starting.
I20260806 22:41:58.964984 16982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:58.965301 16982 log.cc:824] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:58.965837 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.957026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.16.235.125:36009)
0806 22:41:58.957159 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.957162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.957173 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:41:58.957218 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.957219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.957230 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.957302 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.957995 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.958787 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.959686 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.960005 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.960080 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.963056 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.963058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.963058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.963067 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.964646 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.965015 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.965016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.965016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.965080 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.965282 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.965283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.965283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.965442 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.965616 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.965653 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.965731 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 22:41:58.966200 16982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726: No bootstrap required, opened a new log
I20260806 22:41:58.967482 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:41:58.968438 16729 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "92fd2db4e1e04924b5fe070bc92ac581"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58888
I20260806 22:41:58.968421 16982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } }
I20260806 22:41:58.968586 16982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230a87af30124046bf17587ab7b3f726, State: Initialized, Role: FOLLOWER
I20260806 22:41:58.968607 16729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:41:58.968659 16729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:58.968676 16982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } }
I20260806 22:41:58.969051 16982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:58.969003 17013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:58.969102 17013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:41:58.969627 16729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } }
I20260806 22:41:58.969923 16729 leader_election.cc:288] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [CANDIDATE]: Term 1 election: Requested vote from peers 892201bf235045218b84ef06c422d021 (127.16.235.125:36009), 230a87af30124046bf17587ab7b3f726 (127.16.235.124:41797)
W20260806 22:41:58.971354 17025 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:41:58.971419 17025 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:41:58.970485 16948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92fd2db4e1e04924b5fe070bc92ac581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "230a87af30124046bf17587ab7b3f726"
I20260806 22:41:58.972026 16948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:58.973064 16948 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92fd2db4e1e04924b5fe070bc92ac581 in term 1.
I20260806 22:41:58.973394 16690 leader_election.cc:302] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 230a87af30124046bf17587ab7b3f726, 92fd2db4e1e04924b5fe070bc92ac581; no voters:
I20260806 22:41:58.973551 16993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:41:58.973734 16993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [term 1 LEADER]: Becoming Leader. State: Replica: 92fd2db4e1e04924b5fe070bc92ac581, State: Running, Role: LEADER
I20260806 22:41:58.973842 16993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } }
I20260806 22:41:58.979059 16875 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } attempt: 3
I20260806 22:41:58.979029 16885 master_runner.cc:186] Error getting master registration for 127.16.235.125:36009: OK, instance_id { permanent_uuid: "892201bf235045218b84ef06c422d021" instance_seqno: 1786056118742334 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:41:58.982302 16885 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 22:41:58 dist-test-slave-3kk6 krb5kdc[16647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056118, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
I20260806 22:41:58.988665 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.979987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.16.235.124:41797)
0806 22:41:58.980025 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.980027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.980034 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:41:58.980054 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.980054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.980076 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.980125 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.980522 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.981297 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.982263 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.982579 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.982805 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.985301 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.985303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.985303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.985311 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.987432 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.987871 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.987871 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.987871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.987911 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.988091 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.988092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.988093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.988231 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.988296 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.988490 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.988572 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:41:58.988665 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.979735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:41797 (local address 127.0.0.1:42290)
0806 22:41:58.979855 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 22:41:58.979867 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:41:58.979914 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:41:58.980166 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:41:58.980167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:41:58.980368 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:41:58.980415 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.980417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.981406 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.981407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.982157 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:41:58.982160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.982677 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.982678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:41:58.982743 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.983255 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:41:58.983261 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:41:58.984986 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:41:58.987508 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.987509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.987510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.987762 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.987971 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:41:58.987971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:41:58.987972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:41:58.988037 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:41:58.988326 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:41:58.988327 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:41:58.988375 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:41:58.988442 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:41:58.988562 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 22:41:58.991124 16875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021: Bootstrap starting.
I20260806 22:41:58.991982 16875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021: Neither blocks nor log segments found. Creating new log.
I20260806 22:41:58.992332 16875 log.cc:824] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021: Log is configured to *not* fsync() on all Append() calls
I20260806 22:41:58.993363 16875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021: No bootstrap required, opened a new log
I20260806 22:41:58.995679 16875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } }
I20260806 22:41:58.995954 16875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892201bf235045218b84ef06c422d021, State: Initialized, Role: FOLLOWER
I20260806 22:41:58.996122 16875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } }
I20260806 22:41:58.996656 16875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:41:58.996765 16841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92fd2db4e1e04924b5fe070bc92ac581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "892201bf235045218b84ef06c422d021"
I20260806 22:41:58.996881 16841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:41:58.997936 16841 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92fd2db4e1e04924b5fe070bc92ac581 in term 1.
I20260806 22:41:58.998322 17037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92fd2db4e1e04924b5fe070bc92ac581. Latest consensus state: current_term: 1 leader_uuid: "92fd2db4e1e04924b5fe070bc92ac581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:58.998932 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.974991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.16.235.124:41797)
0806 22:41:58.975034 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.975036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.975046 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:41:58.975070 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.975071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.975084 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.975142 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.976050 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.976535 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.977633 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.977945 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.978019 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.984251 (+ 6232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.984252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.984253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.984262 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.986170 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.987763 (+ 1593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.987764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.987765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.987821 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.989454 (+ 1633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.989454 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.989455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.989600 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:58.989676 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:58.998627 (+ 8951us) server_negotiation.cc:299] Negotiation successful
0806 22:41:58.998815 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:41:58.999437 17037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:59.000882 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.975615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.16.235.126:40739)
0806 22:41:58.975667 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.975669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.982846 (+ 7177us) server_negotiation.cc:407] Connection header received
0806 22:41:58.982995 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.982997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.983010 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.983068 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.991355 (+ 8287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.992108 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.993181 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.993478 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.993559 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.995617 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.995618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.995619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.995628 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:58.999165 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.999675 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.999677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.999677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:58.999746 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:58.999960 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:58.999961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:58.999961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:59.000115 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:59.000240 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:59.000670 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 22:41:59.000768 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:41:59.001751 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:41:58.975947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.16.235.125:36009)
0806 22:41:58.975996 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:58.975998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:58.991468 (+ 15470us) server_negotiation.cc:407] Connection header received
0806 22:41:58.991552 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:58.991553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:58.991565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:58.991597 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:58.993264 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.993986 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.996348 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:41:58.996662 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:41:58.996706 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:41:58.998224 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:41:58.998226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:41:58.998226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:41:58.998234 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:41:59.000016 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:59.000447 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:59.000448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:59.000449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:59.000509 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:41:59.000713 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:41:59.000714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:41:59.000714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:41:59.000841 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:41:59.000932 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:41:59.001549 (+ 617us) server_negotiation.cc:299] Negotiation successful
0806 22:41:59.001635 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:41:59.002247 17044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:59.002370 17044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:41:59.007084 17052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:41:59.007337 17052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:41:59.009730 16841 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [term 1 FOLLOWER]: Refusing update from remote peer 92fd2db4e1e04924b5fe070bc92ac581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:59.010040 16948 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [term 1 FOLLOWER]: Refusing update from remote peer 92fd2db4e1e04924b5fe070bc92ac581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:41:59.010380 16993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:41:59.010475 17060 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:41:59.010519 17060 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:41:59.010556 16993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:41:59.011952 17013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92fd2db4e1e04924b5fe070bc92ac581. Latest consensus state: current_term: 1 leader_uuid: "92fd2db4e1e04924b5fe070bc92ac581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:59.012053 17013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:41:59.013489 17044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92fd2db4e1e04924b5fe070bc92ac581. Latest consensus state: current_term: 1 leader_uuid: "92fd2db4e1e04924b5fe070bc92ac581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:59.013588 17044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:41:59.015302 17037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92fd2db4e1e04924b5fe070bc92ac581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:59.015410 17037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:59.015904 17013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92fd2db4e1e04924b5fe070bc92ac581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:59.015996 17013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:41:59.016206 17061 mvcc.cc:205] Tried to move back new op lower bound from 7315685863461298176 to 7315685863317659648. Current Snapshot: MvccSnapshot[applied={T|T < 7315685863461298176}]
I20260806 22:41:59.016399 17063 mvcc.cc:205] Tried to move back new op lower bound from 7315685863461298176 to 7315685863317659648. Current Snapshot: MvccSnapshot[applied={T|T < 7315685863461298176}]
I20260806 22:41:59.019546 17052 catalog_manager.cc:1347] Generated new cluster ID: 46324e4e48fa491ba0aab9d49aa372df
I20260806 22:41:59.019609 17052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:41:59.019829 16993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92fd2db4e1e04924b5fe070bc92ac581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:59.019908 16993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581 [sys.catalog]: This master's current role is: LEADER
I20260806 22:41:59.020340 17044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92fd2db4e1e04924b5fe070bc92ac581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92fd2db4e1e04924b5fe070bc92ac581" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 40739 } } peers { permanent_uuid: "892201bf235045218b84ef06c422d021" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 36009 } } peers { permanent_uuid: "230a87af30124046bf17587ab7b3f726" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 41797 } } }
I20260806 22:41:59.020438 17044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:41:59.020514 17065 mvcc.cc:205] Tried to move back new op lower bound from 7315685863461298176 to 7315685863317659648. Current Snapshot: MvccSnapshot[applied={T|T < 7315685863461298176}]
I20260806 22:41:59.042769 17052 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:41:59.043733 17052 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:41:59.054471 17052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92fd2db4e1e04924b5fe070bc92ac581: Generated new TSK 0
I20260806 22:41:59.054833 17052 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:41:59.083442 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.081954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41394 (local address 127.16.235.125:36009)
0806 22:41:59.082004 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.082007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.082720 (+ 713us) server_negotiation.cc:407] Connection header received
0806 22:41:59.082838 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.082839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.082853 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.082888 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.083367 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41394: BlockingRecv error: recv got EOF from 127.0.0.1:41394 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:41:59.084419 16770 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.082372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.16.235.126:40739)
0806 22:41:59.082417 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.082419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.083899 (+ 1480us) server_negotiation.cc:407] Connection header received
0806 22:41:59.084001 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.084002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.084013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.084044 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.084354 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58894: BlockingRecv error: recv got EOF from 127.0.0.1:58894 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:41:59.085593 16989 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.084934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.16.235.124:41797)
0806 22:41:59.084977 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.084978 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.085090 (+ 112us) server_negotiation.cc:407] Connection header received
0806 22:41:59.085183 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.085183 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.085194 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.085224 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.085532 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42300: BlockingRecv error: recv got EOF from 127.0.0.1:42300 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:41:59.088282 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.087400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.16.235.125:36009)
0806 22:41:59.087447 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.087449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.087719 (+ 270us) server_negotiation.cc:407] Connection header received
0806 22:41:59.087841 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.087842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.087864 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.087896 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.088228 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41396: BlockingRecv error: recv got EOF from 127.0.0.1:41396 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:41:59.095593 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.090720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.16.235.125:36009)
0806 22:41:59.090762 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.090765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.094927 (+ 4162us) server_negotiation.cc:407] Connection header received
0806 22:41:59.095051 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.095052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.095066 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.095099 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.095524 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41410: BlockingRecv error: recv got EOF from 127.0.0.1:41410 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:41:59.100210 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.099162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41414 (local address 127.16.235.125:36009)
0806 22:41:59.099225 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.099227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.099544 (+ 317us) server_negotiation.cc:407] Connection header received
0806 22:41:59.099650 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.099651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.099664 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.099707 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.100150 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41414: BlockingRecv error: recv got EOF from 127.0.0.1:41414 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 22:41:59.106622 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.105662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41418 (local address 127.16.235.125:36009)
0806 22:41:59.105717 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.105719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.106015 (+ 296us) server_negotiation.cc:407] Connection header received
0806 22:41:59.106127 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.106129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.106141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.106172 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.106562 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41418: BlockingRecv error: recv got EOF from 127.0.0.1:41418 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:41:59.117316 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.116232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.16.235.125:36009)
0806 22:41:59.116282 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.116285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.116628 (+ 343us) server_negotiation.cc:407] Connection header received
0806 22:41:59.116760 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.116762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.116776 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.116809 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.117250 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41420: BlockingRecv error: recv got EOF from 127.0.0.1:41420 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:41:59.136098 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.135014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.16.235.125:36009)
0806 22:41:59.135064 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.135067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.135409 (+ 342us) server_negotiation.cc:407] Connection header received
0806 22:41:59.135523 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.135524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.135538 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.135570 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.136031 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41436: BlockingRecv error: recv got EOF from 127.0.0.1:41436 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 22:41:59.170895 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.169787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.16.235.125:36009)
0806 22:41:59.169838 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.169841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.170179 (+ 338us) server_negotiation.cc:407] Connection header received
0806 22:41:59.170292 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.170293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.170306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.170336 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.170828 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41450: BlockingRecv error: recv got EOF from 127.0.0.1:41450 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:41:59.237351 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.236549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41452 (local address 127.16.235.125:36009)
0806 22:41:59.236629 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.236632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.236756 (+ 124us) server_negotiation.cc:407] Connection header received
0806 22:41:59.236815 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.236816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.236828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.236896 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.237276 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41452: BlockingRecv error: recv got EOF from 127.0.0.1:41452 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 22:41:59.367868 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.366912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.16.235.125:36009)
0806 22:41:59.367018 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.367021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.367135 (+ 114us) server_negotiation.cc:407] Connection header received
0806 22:41:59.367245 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.367246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.367267 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.367335 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.367753 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41458: BlockingRecv error: recv got EOF from 127.0.0.1:41458 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 22:41:59.626189 16882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:41:59.625431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41470 (local address 127.16.235.125:36009)
0806 22:41:59.625514 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:41:59.625517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:41:59.625530 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:41:59.625595 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:41:59.625597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:41:59.625613 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:41:59.625690 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:41:59.626098 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41470: BlockingRecv error: recv got EOF from 127.0.0.1:41470 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:41:59.971966 17025 catalog_manager.cc:1259] Loaded cluster ID: 46324e4e48fa491ba0aab9d49aa372df
I20260806 22:41:59.972064 17025 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726: loading cluster ID for follower catalog manager: success
I20260806 22:41:59.972942 17025 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726: acquiring CA information for follower catalog manager: success
I20260806 22:41:59.973300 17025 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 230a87af30124046bf17587ab7b3f726: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:42:00.011066 17060 catalog_manager.cc:1259] Loaded cluster ID: 46324e4e48fa491ba0aab9d49aa372df
I20260806 22:42:00.011178 17060 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021: loading cluster ID for follower catalog manager: success
I20260806 22:42:00.012001 17060 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021: acquiring CA information for follower catalog manager: success
I20260806 22:42:00.012313 17060 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 892201bf235045218b84ef06c422d021: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:42:00.142418 17173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:00.139663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41482 (local address 127.16.235.125:36009)
0806 22:42:00.139788 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:00.139790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:00.139823 (+ 33us) server_negotiation.cc:407] Connection header received
0806 22:42:00.139901 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:00.139902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:00.139961 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:00.140055 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:00.140299 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.140840 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.141605 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.141811 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.141869 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:00.142037 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:00.142150 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:00.142212 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:00.142253 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 22:42:00.142272 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260806 22:42:00.147385 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:00.144548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.16.235.124:41797)
0806 22:42:00.144680 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:00.144682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:00.144691 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:00.144733 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:00.144734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:00.144774 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:00.144846 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:00.144958 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.145448 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.146278 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.146596 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.146710 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:00.146912 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:00.147061 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:00.147139 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:00.147189 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 22:42:00.147212 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260806 22:42:00.152593 17173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:00.149361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.16.235.125:36009)
0806 22:42:00.149428 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:00.149430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:00.149671 (+ 241us) server_negotiation.cc:407] Connection header received
0806 22:42:00.149721 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:00.149722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:00.149735 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:00.149796 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:00.149932 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.150759 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.151870 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.152077 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.152112 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:00.152298 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:00.152335 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:00.152391 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:00.152460 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 22:42:00.152484 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:42:00.152801 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:00.149360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.16.235.126:40739)
0806 22:42:00.149587 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:00.149589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:00.149853 (+ 264us) server_negotiation.cc:407] Connection header received
0806 22:42:00.149934 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:00.149935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:00.149992 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:00.150097 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:00.150234 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.150990 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.151740 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.152052 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.152119 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:00.152366 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:00.152481 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:00.152534 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:00.152690 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 22:42:00.152716 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":57,"threads_started":1}
I20260806 22:42:00.153666 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:00.149442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.16.235.124:41797)
0806 22:42:00.149480 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:00.149482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:00.149724 (+ 242us) server_negotiation.cc:407] Connection header received
0806 22:42:00.149845 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:00.149846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:00.149857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:00.149916 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:00.151005 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.151745 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.152758 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:00.153036 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:00.153113 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:00.153413 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:00.153450 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:00.153486 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:00.153556 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 22:42:00.153580 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 22:42:00.154903 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 16667
I20260806 22:42:00.169361 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 16775
I20260806 22:42:00.179723 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 16885
2026-08-06T22:42:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3089 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:42:00 dist-test-slave-3kk6 krb5kdc[17200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:42:00 dist-test-slave-3kk6 krb5kdc[17200](info): set up 2 sockets
Aug 06 22:42:00 dist-test-slave-3kk6 krb5kdc[17200](info): commencing operation
krb5kdc: starting...
W20260806 22:42:01.417865 17325 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/test-user.keytab.
Aug 06 22:42:01 dist-test-slave-3kk6 krb5kdc[17200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:42:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:42:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.235.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
Entry for principal kudu/127.16.235.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab.
I20260806 22:42:01.549939 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45741
--webserver_interface=127.16.235.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:45741,127.16.235.125:37399,127.16.235.124:42751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:42:01.636067 17217 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:42:01.636332 17217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:01.636351 17217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:01.636373 17217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:01.637574 17217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:01.637622 17217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:01.637636 17217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:01.637648 17217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:01.637661 17217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:01.639200 17217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:45741,127.16.235.125:37399,127.16.235.124:42751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.126.keytab
--principal=kudu/127.16.235.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.126:45741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.235.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:01.639422 17217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:01.639612 17217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:01.644889 17224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:01.644910 17223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:01.644925 17217 server_base.cc:1034] running on GCE node
W20260806 22:42:01.644881 17227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:01.645308 17217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:01.645637 17217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:01.646829 17217 hybrid_clock.cc:648] HybridClock initialized: now 1786056121646813 us; error 39 us; skew 500 ppm
Aug 06 22:42:01 dist-test-slave-3kk6 krb5kdc[17200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:01.649649 17217 init.cc:377] Logged in from keytab as kudu/127.16.235.126@KRBTEST.COM (short username kudu)
I20260806 22:42:01.650209 17217 webserver.cc:460] Webserver started at http://127.16.235.126:45695/ using document root <none> and password file <none>
I20260806 22:42:01.650446 17217 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:01.650492 17217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:01.650700 17217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:01.651871 17217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/data/instance:
uuid: "e2b2cd90414f403e89ecec8ef250d670"
format_stamp: "Formatted at 2026-08-06 22:42:01 on dist-test-slave-3kk6"
server_key: "eab8c632ebf7edb8f9f68863696e9e83"
server_key_iv: "1a133d4a0328b52e874dcc005257b695"
server_key_version: "encryptionkey@0"
I20260806 22:42:01.652266 17217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/wal/instance:
uuid: "e2b2cd90414f403e89ecec8ef250d670"
format_stamp: "Formatted at 2026-08-06 22:42:01 on dist-test-slave-3kk6"
server_key: "eab8c632ebf7edb8f9f68863696e9e83"
server_key_iv: "1a133d4a0328b52e874dcc005257b695"
server_key_version: "encryptionkey@0"
I20260806 22:42:01.653748 17217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:42:01.654561 17236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:01.654781 17217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:01.654892 17217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/wal
uuid: "e2b2cd90414f403e89ecec8ef250d670"
format_stamp: "Formatted at 2026-08-06 22:42:01 on dist-test-slave-3kk6"
server_key: "eab8c632ebf7edb8f9f68863696e9e83"
server_key_iv: "1a133d4a0328b52e874dcc005257b695"
server_key_version: "encryptionkey@0"
I20260806 22:42:01.654971 17217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:01.674304 17217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:01.676683 17217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:01.676870 17217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:01.681556 17217 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.126:45741
I20260806 22:42:01.681597 17306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.126:45741 every 8 connection(s)
I20260806 22:42:01.682014 17217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/data/info.pb
I20260806 22:42:01.682997 17307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:01.684605 17307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:01.686491 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 17217
I20260806 22:42:01.686616 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-0/wal/instance
I20260806 22:42:01.686853 17325 external_mini_cluster.cc:1428] Setting key c092ec18c1ddc792d3dca2494344b4a9
Aug 06 22:42:01 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.235.125@KRBTEST.COM; defaulting to no policy
I20260806 22:42:01.695226 17316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.685369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33978 (local address 127.16.235.126:45741)
0806 22:42:01.685538 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:01.685548 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:01.685678 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:42:01.686281 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:01.686283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:01.686302 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:01.686406 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:01.687195 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.688045 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.688997 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.689310 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.689432 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:01.691743 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:01.691750 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:01.691751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:01.691760 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:01.693453 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:01.693874 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:01.693878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:01.693879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:01.693965 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:01.694211 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:01.694212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:01.694212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:01.694463 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:01.694563 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:01.694795 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 22:42:01.694953 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260806 22:42:01.695225 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.685293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45741 (local address 127.0.0.1:33978)
0806 22:42:01.685527 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.685576 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:01.686108 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:01.686415 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:01.686417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:01.686827 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:01.687064 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:01.687070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.688179 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.688180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:01.688899 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:01.688901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.689403 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.689403 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:01.689472 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:01.690112 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:01.690122 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:01.691614 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:01.693561 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:01.693568 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:01.693571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:01.693780 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:01.694039 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:01.694039 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:01.694040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:01.694132 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:01.694541 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:01.694547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:01.694614 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 22:42:01.694760 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 22:42:01.695024 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":104,"thread_start_us":106,"threads_started":1}
I20260806 22:42:01.696100 17307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:01.696503 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.696380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:37399 (local address 127.0.0.1:53136)
0806 22:42:01.696433 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.696481 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:42:01.696659 17240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
I20260806 22:42:01.697125 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.697045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:37399 (local address 127.0.0.1:53138)
0806 22:42:01.697085 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.697109 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:42:01.697249 17307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.125:37399: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab.
Entry for principal kudu/127.16.235.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab.
I20260806 22:42:01.714190 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.125:37399
--webserver_interface=127.16.235.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab
--principal=kudu/127.16.235.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:45741,127.16.235.125:37399,127.16.235.124:42751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:01.746529 17307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } attempt: 1
I20260806 22:42:01.747156 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.747008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:37399 (local address 127.0.0.1:53140)
0806 22:42:01.747081 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.747125 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:42:01.747537 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.747432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:37399 (local address 127.0.0.1:53146)
0806 22:42:01.747481 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.747507 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:42:01.747653 17307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.125:37399: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
W20260806 22:42:01.802047 17322 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:42:01.802294 17322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:01.802311 17322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:01.802333 17322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:01.803598 17322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:01.803651 17322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:01.803666 17322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:01.803679 17322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:01.803740 17322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:01.805317 17322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:45741,127.16.235.125:37399,127.16.235.124:42751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.125.keytab
--principal=kudu/127.16.235.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.125:37399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.235.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:01.805600 17322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:01.805864 17322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:01.811302 17336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:01.811331 17338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:01.811302 17335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:01.811610 17322 server_base.cc:1034] running on GCE node
I20260806 22:42:01.811894 17322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:01.812307 17322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:01.813488 17322 hybrid_clock.cc:648] HybridClock initialized: now 1786056121813460 us; error 51 us; skew 500 ppm
I20260806 22:42:01.815797 17307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } attempt: 2
Aug 06 22:42:01 dist-test-slave-3kk6 krb5kdc[17200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:01.816286 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.816135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:37399 (local address 127.0.0.1:53156)
0806 22:42:01.816216 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.816255 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:42:01.816435 17322 init.cc:377] Logged in from keytab as kudu/127.16.235.125@KRBTEST.COM (short username kudu)
I20260806 22:42:01.816658 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.816551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:37399 (local address 127.0.0.1:53164)
0806 22:42:01.816601 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.816630 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 22:42:01.816783 17307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.125:37399: Network error: Client connection negotiation failed: client connection to 127.16.235.125:37399: connect: Connection refused (error 111)
I20260806 22:42:01.816995 17322 webserver.cc:460] Webserver started at http://127.16.235.125:43267/ using document root <none> and password file <none>
I20260806 22:42:01.817247 17322 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:01.817293 17322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:01.817451 17322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:01.818599 17322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/data/instance:
uuid: "a7c3d9da99be4beabe25f16c53b644bb"
format_stamp: "Formatted at 2026-08-06 22:42:01 on dist-test-slave-3kk6"
server_key: "1da1b4383fd21cc873358c4630e29801"
server_key_iv: "66273d5a8174d8b0176d6e2f7c859e71"
server_key_version: "encryptionkey@0"
I20260806 22:42:01.819079 17322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/wal/instance:
uuid: "a7c3d9da99be4beabe25f16c53b644bb"
format_stamp: "Formatted at 2026-08-06 22:42:01 on dist-test-slave-3kk6"
server_key: "1da1b4383fd21cc873358c4630e29801"
server_key_iv: "66273d5a8174d8b0176d6e2f7c859e71"
server_key_version: "encryptionkey@0"
I20260806 22:42:01.820487 17322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:42:01.821343 17349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:01.821550 17322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:42:01.821655 17322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/wal
uuid: "a7c3d9da99be4beabe25f16c53b644bb"
format_stamp: "Formatted at 2026-08-06 22:42:01 on dist-test-slave-3kk6"
server_key: "1da1b4383fd21cc873358c4630e29801"
server_key_iv: "66273d5a8174d8b0176d6e2f7c859e71"
server_key_version: "encryptionkey@0"
I20260806 22:42:01.821717 17322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:01.838459 17322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:01.841105 17322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:01.841246 17322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:01.845536 17322 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.125:37399
I20260806 22:42:01.845577 17414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.125:37399 every 8 connection(s)
I20260806 22:42:01.845973 17322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/data/info.pb
I20260806 22:42:01.847026 17415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:01.848626 17415 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:01.850740 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 17322
I20260806 22:42:01.850876 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-1/wal/instance
I20260806 22:42:01.851081 17325 external_mini_cluster.cc:1428] Setting key 378b9e1215f836e2591fa66c1ac8b22b
Aug 06 22:42:01 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:42:01.859601 17316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.849391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.16.235.126:45741)
0806 22:42:01.849471 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:01.849473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:01.849686 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:42:01.850524 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:01.850526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:01.850543 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:01.850673 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:01.851500 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.852258 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.853296 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.853522 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.853617 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:01.855972 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:01.855974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:01.855974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:01.855984 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:01.857828 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:01.858286 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:01.858287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:01.858287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:01.858355 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:01.858789 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:01.858790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:01.858791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:01.858962 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:01.859041 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:01.859377 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 22:42:01.859475 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:42:01.859639 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.849346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45741 (local address 127.0.0.1:33988)
0806 22:42:01.849525 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.849570 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:01.850342 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:01.850800 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:01.850805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:01.851137 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:01.851397 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:01.851401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.852366 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.852367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:01.853168 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:01.853170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.853630 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.853631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:01.853774 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:01.854365 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:01.854373 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:01.855868 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:01.857936 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:01.857944 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:01.857946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:01.858200 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:01.858453 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:01.858454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:01.858454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:01.858710 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:01.859058 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:01.859070 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:01.859208 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 22:42:01.859303 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:42:01.859454 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260806 22:42:01.860399 17415 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.235.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.235.124@KRBTEST.COM" created.
Aug 06 22:42:01 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
I20260806 22:42:01.868394 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.860717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:37399 (local address 127.0.0.1:53180)
0806 22:42:01.860925 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.860935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:01.860974 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:01.861169 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:01.861170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:01.861289 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:01.861342 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:01.861344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.862235 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.862236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:01.862823 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:01.862826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.863283 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.863284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:01.863343 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:01.863897 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:01.863902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:01.865125 (+ 1223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:01.867256 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:01.867257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:01.867257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:01.867481 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:01.867736 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:01.867737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:01.867737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:01.867794 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:01.868140 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:01.868141 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:01.868177 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:42:01.868239 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:42:01.868324 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260806 22:42:01.868566 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.860783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53180 (local address 127.16.235.125:37399)
0806 22:42:01.860970 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:01.860975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:01.861030 (+ 55us) server_negotiation.cc:407] Connection header received
0806 22:42:01.861059 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:01.861061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:01.861077 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:01.861152 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:01.861410 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.862152 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.862908 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.863212 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.863270 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:01.865227 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:01.865240 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:01.865241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:01.865246 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:01.867092 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:01.867583 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:01.867587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:01.867587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:01.867660 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:01.867861 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:01.867861 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:01.867862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:01.868069 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:01.868158 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:01.868271 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:42:01.868399 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
I20260806 22:42:01.869369 17415 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:01.869751 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.869629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58296)
0806 22:42:01.869680 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.869731 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:42:01.869889 17350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
I20260806 22:42:01.870373 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.870281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58302)
0806 22:42:01.870327 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.870352 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:42:01.870491 17415 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:42751: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.235.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab.
Entry for principal kudu/127.16.235.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab.
I20260806 22:42:01.880837 17325 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVIGsRX/build/release/bin/kudu
/tmp/dist-test-taskVIGsRX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.124:42751
--webserver_interface=127.16.235.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab
--principal=kudu/127.16.235.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.235.84:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.235.126:45741,127.16.235.125:37399,127.16.235.124:42751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:42:01.907961 17307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } attempt: 3
Aug 06 22:42:01 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
I20260806 22:42:01.916095 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.908533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.16.235.125:37399)
0806 22:42:01.908609 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:01.908611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:01.908625 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:42:01.908704 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:01.908706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:01.908722 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:01.908807 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:01.909163 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.909873 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.910608 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.910899 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.910985 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:01.913149 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:01.913151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:01.913151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:01.913158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:01.914826 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:01.915273 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:01.915275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:01.915275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:01.915353 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:01.915585 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:01.915586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:01.915586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:01.915729 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:01.915786 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:01.915951 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 22:42:01.916022 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:42:01.916134 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.908451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:37399 (local address 127.0.0.1:53190)
0806 22:42:01.908528 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.908548 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:01.908630 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:01.908813 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:01.908814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:01.909002 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:01.909070 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:01.909072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.909965 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.909966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:01.910510 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:01.910512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:01.910982 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:01.910984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:01.911075 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:01.911616 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:01.911621 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:01.913035 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:01.914909 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:01.914910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:01.914911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:01.915141 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:01.915437 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:01.915438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:01.915439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:01.915516 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:01.915796 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:01.915797 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:01.915850 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:42:01.915923 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:42:01.916035 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 22:42:01.916699 17307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:01.917088 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.916966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58308)
0806 22:42:01.917020 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.917061 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:42:01.917472 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.917377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58314)
0806 22:42:01.917426 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.917455 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:42:01.917600 17307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:42751: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
I20260806 22:42:01.919713 17415 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } attempt: 1
I20260806 22:42:01.920080 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.919968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58316)
0806 22:42:01.920023 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.920057 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:42:01.920394 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.920297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58330)
0806 22:42:01.920342 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.920372 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:42:01.920485 17415 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:42751: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
I20260806 22:42:01.948786 17307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } attempt: 1
I20260806 22:42:01.949505 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.949310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58338)
0806 22:42:01.949399 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.949459 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 22:42:01.950021 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.949860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58340)
0806 22:42:01.949951 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.949991 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260806 22:42:01.950179 17307 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:42751: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
W20260806 22:42:01.969951 17428 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:42:01.970189 17428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:42:01.970207 17428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:42:01.970229 17428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:42:01.971367 17428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:42:01.971411 17428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:42:01.971427 17428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:42:01.971441 17428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:42:01.971452 17428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:42:01.973006 17428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.235.84:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.235.126:45741,127.16.235.125:37399,127.16.235.124:42751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/krb5kdc/kudu_127.16.235.124.keytab
--principal=kudu/127.16.235.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.235.124:42751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.235.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:30:49 UTC on ccycloud.quasar-lekpjo.root.comops.site
build id 4571391
I20260806 22:42:01.973227 17428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:42:01.973410 17428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:42:01.978464 17436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:01.978523 17439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:42:01.978484 17443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:42:01.978828 17428 server_base.cc:1034] running on GCE node
I20260806 22:42:01.979075 17428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:42:01.979346 17428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:42:01.980494 17428 hybrid_clock.cc:648] HybridClock initialized: now 1786056121980482 us; error 28 us; skew 500 ppm
Aug 06 22:42:01 dist-test-slave-3kk6 krb5kdc[17200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:42:01.983125 17428 init.cc:377] Logged in from keytab as kudu/127.16.235.124@KRBTEST.COM (short username kudu)
I20260806 22:42:01.983616 17428 webserver.cc:460] Webserver started at http://127.16.235.124:33361/ using document root <none> and password file <none>
I20260806 22:42:01.983856 17428 fs_manager.cc:359] Metadata directory not provided
I20260806 22:42:01.983929 17428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:42:01.984092 17428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:42:01.985236 17428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/data/instance:
uuid: "72dfe5c133644a36abc87f4972c851bc"
format_stamp: "Formatted at 2026-08-06 22:42:01 on dist-test-slave-3kk6"
server_key: "d0a7a377b0e63a2dab6b8e6806e700ac"
server_key_iv: "3dcbf2aa539d357757319a3241a0a3ed"
server_key_version: "encryptionkey@0"
I20260806 22:42:01.985633 17428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/wal/instance:
uuid: "72dfe5c133644a36abc87f4972c851bc"
format_stamp: "Formatted at 2026-08-06 22:42:01 on dist-test-slave-3kk6"
server_key: "d0a7a377b0e63a2dab6b8e6806e700ac"
server_key_iv: "3dcbf2aa539d357757319a3241a0a3ed"
server_key_version: "encryptionkey@0"
I20260806 22:42:01.987179 17428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 22:42:01.987920 17451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:42:01.988087 17428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 22:42:01.988204 17428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/data,/tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/wal
uuid: "72dfe5c133644a36abc87f4972c851bc"
format_stamp: "Formatted at 2026-08-06 22:42:01 on dist-test-slave-3kk6"
server_key: "d0a7a377b0e63a2dab6b8e6806e700ac"
server_key_iv: "3dcbf2aa539d357757319a3241a0a3ed"
server_key_version: "encryptionkey@0"
I20260806 22:42:01.988283 17428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:42:01.988626 17415 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } attempt: 2
I20260806 22:42:01.989055 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.988917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58344)
0806 22:42:01.988980 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.989015 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:42:01.989419 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:01.989308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58350)
0806 22:42:01.989360 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:01.989392 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:42:01.989506 17415 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.235.124:42751: Network error: Client connection negotiation failed: client connection to 127.16.235.124:42751: connect: Connection refused (error 111)
I20260806 22:42:02.015781 17428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:42:02.017895 17428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:42:02.018081 17428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:42:02.022508 17428 rpc_server.cc:307] RPC server started. Bound to: 127.16.235.124:42751
I20260806 22:42:02.022544 17526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.235.124:42751 every 8 connection(s)
I20260806 22:42:02.022992 17428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/data/info.pb
I20260806 22:42:02.024149 17527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:42:02.025350 17307 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } attempt: 2
I20260806 22:42:02.026304 17527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:02.027221 17325 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVIGsRX/build/release/bin/kudu as pid 17428
I20260806 22:42:02.027376 17325 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVIGsRX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786055836003438-17325-0/minicluster-data/master-2/wal/instance
I20260806 22:42:02.027639 17325 external_mini_cluster.cc:1428] Setting key fa8d895d9acc10078141a4422ccd2a86
Aug 06 22:42:02 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
Aug 06 22:42:02 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.126@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
Aug 06 22:42:02 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
I20260806 22:42:02.037465 17316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.027541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.16.235.126:45741)
0806 22:42:02.027589 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.027591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.027847 (+ 256us) server_negotiation.cc:407] Connection header received
0806 22:42:02.027968 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.027970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.027985 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.028058 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.028796 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.029391 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.030436 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.030760 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.030853 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.033748 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.033749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.033750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.033757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.035480 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.036160 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.036162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.036162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.036238 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.036608 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.036609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.036610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.036851 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.036922 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.037271 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.037366 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:42:02.037667 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.027499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.126:45741 (local address 127.0.0.1:33990)
0806 22:42:02.027747 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:02.027787 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:02.027876 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:02.028143 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:02.028148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:02.028584 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:02.028694 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.028701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.029494 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.029494 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.030339 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.030341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.030924 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.030925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.031072 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.031845 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:02.031857 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:02.033653 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:02.035575 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.035584 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.035587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.036076 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.036318 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.036319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.036319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.036508 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.036951 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:02.036957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:02.037100 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 22:42:02.037215 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:42:02.037408 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
I20260806 22:42:02.038412 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.025875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58352)
0806 22:42:02.025945 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:02.025959 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:02.026012 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:02.027215 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:02.027217 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:02.027423 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:02.027496 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.027500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.029951 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.029952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.030561 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.030562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.031130 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.031131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.031198 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.031933 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:02.031937 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:02.034214 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:02.036928 (+ 2714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.036929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.036931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.037223 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.037511 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.037512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.037512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.037607 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.038043 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:02.038044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:02.038103 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 22:42:02.038187 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:42:02.038313 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 22:42:02.038605 17529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.026068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58352 (local address 127.16.235.124:42751)
0806 22:42:02.026307 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.026315 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.026346 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:42:02.026940 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.026949 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.027017 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.027226 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.027627 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.029152 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.030674 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.031042 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.031129 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.034618 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.034627 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.034627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.034635 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.036824 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.037341 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.037345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.037346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.037415 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.037691 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.037691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.037692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.037900 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.038017 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.038251 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.038452 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":108,"threads_started":1}
I20260806 22:42:02.038825 17527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:02.039575 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.028003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58360 (local address 127.16.235.124:42751)
0806 22:42:02.028132 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.028133 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.028330 (+ 197us) server_negotiation.cc:407] Connection header received
0806 22:42:02.028393 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.028394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.028405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.028495 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.029243 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.029859 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.030823 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.031167 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.031249 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.035136 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.035140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.035140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.035150 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.037798 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.038543 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.038545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.038546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.038625 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.038897 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.038898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.038898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.039037 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.039177 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.039319 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.039416 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260806 22:42:02.040723 17307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670: Bootstrap starting.
I20260806 22:42:02.041540 17307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:02.041882 17307 log.cc:824] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:02.043028 17307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670: No bootstrap required, opened a new log
Aug 06 22:42:02 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
Aug 06 22:42:02 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.126@KRBTEST.COM
I20260806 22:42:02.045535 17307 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } }
I20260806 22:42:02.045704 17307 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b2cd90414f403e89ecec8ef250d670, State: Initialized, Role: FOLLOWER
I20260806 22:42:02.045789 17307 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } }
I20260806 22:42:02.046208 17307 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:02.046708 17536 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.046820 17536 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:42:02.048537 17548 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 22:42:02.048640 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.039232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53196 (local address 127.16.235.125:37399)
0806 22:42:02.039280 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.039282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.039362 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:42:02.039462 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.039463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.039474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.039505 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.039925 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.040633 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.041615 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.041948 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.041999 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.044457 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.044459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.044459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.044468 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.047161 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.047657 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.047659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.047660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.047733 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.048008 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.048009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.048009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.048155 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.048201 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.048438 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.048533 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 22:42:02.048650 17548 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:02.049093 17316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.040196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.16.235.126:45741)
0806 22:42:02.040240 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.040243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.040290 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:42:02.040371 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.040372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.040383 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.040445 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.040986 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.041593 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.042494 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.042869 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.042909 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.045346 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.045348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.045348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.045356 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.047264 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.047745 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.047746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.047747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.047816 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.048038 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.048039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.048040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.048218 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.048359 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.048887 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.048983 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:42:02.049243 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.039114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.125:37399 (local address 127.0.0.1:53196)
0806 22:42:02.039309 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:02.039322 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:02.039389 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:02.039570 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:02.039571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:02.039742 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:02.039816 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.039818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.040741 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.040742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.041498 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.041502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.042093 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.042094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.042176 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.042828 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:02.042834 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:02.044335 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:02.047273 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.047274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.047275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.047549 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.047842 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.047843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.047843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.047927 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.048257 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:02.048258 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:02.048314 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:42:02.049011 (+ 697us) client_negotiation.cc:240] Negotiation successful
0806 22:42:02.049145 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260806 22:42:02.049654 17527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:42:02.049641 17217 master_runner.cc:186] Error getting master registration for 127.16.235.125:37399: OK, instance_id { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" instance_seqno: 1786056121844252 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:42:02.052403 17217 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 22:42:02 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.124@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
Aug 06 22:42:02 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.235.125@KRBTEST.COM
I20260806 22:42:02.061506 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.049970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58374)
0806 22:42:02.050017 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:02.050028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:02.050063 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:02.052729 (+ 2666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:02.052730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:02.052931 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:02.052981 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.052983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.053871 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.053873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.054781 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.054784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.055306 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.055307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.055373 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.055955 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:02.055961 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:02.057503 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:02.059562 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.059563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.059564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.060091 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.060593 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.060594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.060594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.060670 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.061104 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:02.061105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:02.061157 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:42:02.061237 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:42:02.061362 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:42:02.061595 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.052516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.16.235.124:42751)
0806 22:42:02.052593 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.052595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.052607 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:42:02.052633 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.052634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.052648 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.052727 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.053129 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.053777 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.054893 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.055212 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.055290 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.057601 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.057602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.057603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.057611 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.059461 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.060180 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.060182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.060182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.060256 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.060766 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.060767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.060767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.061010 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.061102 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.061281 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.061437 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":48}
I20260806 22:42:02.062844 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.053132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53200 (local address 127.16.235.125:37399)
0806 22:42:02.053182 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.053184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.053224 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:42:02.053292 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.053294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.053305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.053373 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.054020 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.054829 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.055738 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.056099 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.056185 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.059482 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.059483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.059484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.059493 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.061337 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.061758 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.061759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.061760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.061821 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.062177 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.062178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.062179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.062328 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.062533 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.062573 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.062695 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:42:02.063592 17527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc: Bootstrap starting.
I20260806 22:42:02.064488 17527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:02.064556 17325 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:42:02.064806 17527 log.cc:824] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:02.065531 17271 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e2b2cd90414f403e89ecec8ef250d670"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34000
I20260806 22:42:02.065649 17527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc: No bootstrap required, opened a new log
I20260806 22:42:02.065701 17271 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:42:02.065759 17271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:02.066753 17271 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } }
I20260806 22:42:02.067029 17271 leader_election.cc:288] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [CANDIDATE]: Term 1 election: Requested vote from peers a7c3d9da99be4beabe25f16c53b644bb (127.16.235.125:37399), 72dfe5c133644a36abc87f4972c851bc (127.16.235.124:42751)
I20260806 22:42:02.067378 17527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } }
I20260806 22:42:02.067538 17527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72dfe5c133644a36abc87f4972c851bc, State: Initialized, Role: FOLLOWER
I20260806 22:42:02.067620 17527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } }
I20260806 22:42:02.067942 17527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:02.068014 17491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e2b2cd90414f403e89ecec8ef250d670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "72dfe5c133644a36abc87f4972c851bc"
I20260806 22:42:02.068105 17491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:02.069644 17491 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b2cd90414f403e89ecec8ef250d670 in term 1.
I20260806 22:42:02.069922 17237 leader_election.cc:302] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72dfe5c133644a36abc87f4972c851bc, e2b2cd90414f403e89ecec8ef250d670; no voters:
I20260806 22:42:02.067905 17552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.070161 17536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:42:02.070353 17536 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [term 1 LEADER]: Becoming Leader. State: Replica: e2b2cd90414f403e89ecec8ef250d670, State: Running, Role: LEADER
I20260806 22:42:02.070442 17536 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } }
I20260806 22:42:02.070062 17552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:42:02.071163 17571 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:02.071209 17571 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:02.071825 17536 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2b2cd90414f403e89ecec8ef250d670. Latest consensus state: current_term: 1 leader_uuid: "e2b2cd90414f403e89ecec8ef250d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.071911 17536 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:02.074292 17428 master_runner.cc:186] Error getting master registration for 127.16.235.125:37399: OK, instance_id { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" instance_seqno: 1786056121844252 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:42:02.077303 17428 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:42:02.078326 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.068617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.16.235.124:42751)
0806 22:42:02.069416 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.069419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.070232 (+ 813us) server_negotiation.cc:407] Connection header received
0806 22:42:02.070345 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.070346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.070358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.070393 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.071301 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.072104 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.073227 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.073507 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.073548 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.075234 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.075235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.075236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.075243 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.076960 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.077382 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.077384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.077384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.077445 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.077662 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.077663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.077664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.077828 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.077935 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.078130 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.078229 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776}
I20260806 22:42:02.079366 17581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:42:02.079623 17581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:42:02.080667 17415 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } attempt: 3
Aug 06 22:42:02 dist-test-slave-3kk6 krb5kdc[17200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056121, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.235.125@KRBTEST.COM for kudu/127.16.235.124@KRBTEST.COM
I20260806 22:42:02.086261 17316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.069949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34008 (local address 127.16.235.126:45741)
0806 22:42:02.069998 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.070001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.071466 (+ 1465us) server_negotiation.cc:407] Connection header received
0806 22:42:02.072402 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.072403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.072415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.072445 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.074902 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.075600 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.077528 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.077794 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.077827 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.080390 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.080391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.080391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.080400 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.083181 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.084445 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.084446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.084446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.084507 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.085334 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.085335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.085335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.085478 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.085561 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.086079 (+ 518us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.086166 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:42:02.086688 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.070193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.16.235.125:37399)
0806 22:42:02.070237 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.070239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.071372 (+ 1133us) server_negotiation.cc:407] Connection header received
0806 22:42:02.071569 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.071570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.071581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.071610 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.073167 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.073844 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.076647 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.076995 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.077040 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.079447 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.079448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.079449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.079456 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.082346 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.084240 (+ 1894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.084241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.084242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.084318 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.085008 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.085009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.085010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.085161 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.085245 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.086496 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.086577 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:42:02.090076 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.080911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.235.124:42751 (local address 127.0.0.1:58388)
0806 22:42:02.081019 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 22:42:02.081032 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:42:02.081074 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:42:02.081366 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:42:02.081367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:42:02.081531 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:42:02.081832 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.081836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.082808 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.082809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.083601 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:42:02.083604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.084116 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.084117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:42:02.084176 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.084589 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:42:02.084594 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:42:02.086199 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:42:02.088772 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.088773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.088773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.088999 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.089258 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:42:02.089259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:42:02.089259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:42:02.089363 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:42:02.089686 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:42:02.089687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:42:02.089778 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 22:42:02.089863 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:42:02.089976 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 22:42:02.090456 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:02.081147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58388 (local address 127.16.235.124:42751)
0806 22:42:02.081218 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.081220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.081229 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:42:02.081250 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.081251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.081261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.081318 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.081923 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.082708 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.083698 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:02.084005 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:02.084063 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:02.086295 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:42:02.086297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:42:02.086297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:42:02.086509 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:42:02.088666 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.089078 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.089079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.089079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.089166 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:42:02.089441 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:42:02.089443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:42:02.089443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:42:02.089591 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:42:02.089656 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:02.090280 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 22:42:02.090366 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:42:02.092514 17491 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [term 1 FOLLOWER]: Refusing update from remote peer e2b2cd90414f403e89ecec8ef250d670: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:02.092593 17415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb: Bootstrap starting.
I20260806 22:42:02.093026 17574 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:02.093595 17415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb: Neither blocks nor log segments found. Creating new log.
I20260806 22:42:02.093920 17415 log.cc:824] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:42:02.094728 17552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2b2cd90414f403e89ecec8ef250d670. Latest consensus state: current_term: 1 leader_uuid: "e2b2cd90414f403e89ecec8ef250d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.094820 17552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:02.096635 17552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2b2cd90414f403e89ecec8ef250d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.096726 17552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:02.097047 17536 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2b2cd90414f403e89ecec8ef250d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.097301 17536 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:02.097692 17584 mvcc.cc:205] Tried to move back new op lower bound from 7315685876086370304 to 7315685876004777984. Current Snapshot: MvccSnapshot[applied={T|T < 7315685876086370304}]
I20260806 22:42:02.099126 17581 catalog_manager.cc:1347] Generated new cluster ID: 8a87baed082d415495c3df4979ab969a
I20260806 22:42:02.099184 17581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:42:02.111578 17415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb: No bootstrap required, opened a new log
I20260806 22:42:02.113966 17415 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } }
I20260806 22:42:02.114147 17415 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7c3d9da99be4beabe25f16c53b644bb, State: Initialized, Role: FOLLOWER
I20260806 22:42:02.114231 17415 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } }
I20260806 22:42:02.114630 17415 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:42:02.114765 17381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:42:02.115559 17581 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:42:02.115797 17381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [term 1 FOLLOWER]: Refusing update from remote peer e2b2cd90414f403e89ecec8ef250d670: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:42:02.116355 17581 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:42:02.116714 17382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e2b2cd90414f403e89ecec8ef250d670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a7c3d9da99be4beabe25f16c53b644bb"
I20260806 22:42:02.117671 17382 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b2cd90414f403e89ecec8ef250d670 in term 1.
I20260806 22:42:02.117960 17587 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:42:02.119699 17608 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2b2cd90414f403e89ecec8ef250d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.119794 17608 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:02.119848 17608 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2b2cd90414f403e89ecec8ef250d670. Latest consensus state: current_term: 1 leader_uuid: "e2b2cd90414f403e89ecec8ef250d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.119899 17608 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:02.119930 17608 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2b2cd90414f403e89ecec8ef250d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.119973 17608 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:42:02.120167 17609 mvcc.cc:205] Tried to move back new op lower bound from 7315685876171681792 to 7315685876004777984. Current Snapshot: MvccSnapshot[applied={T|T < 7315685876086370304}]
W20260806 22:42:02.121268 17606 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:42:02.121330 17606 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:42:02.147725 17587 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2b2cd90414f403e89ecec8ef250d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b2cd90414f403e89ecec8ef250d670" member_type: VOTER last_known_addr { host: "127.16.235.126" port: 45741 } } peers { permanent_uuid: "a7c3d9da99be4beabe25f16c53b644bb" member_type: VOTER last_known_addr { host: "127.16.235.125" port: 37399 } } peers { permanent_uuid: "72dfe5c133644a36abc87f4972c851bc" member_type: VOTER last_known_addr { host: "127.16.235.124" port: 42751 } } }
I20260806 22:42:02.147890 17587 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670 [sys.catalog]: This master's current role is: LEADER
I20260806 22:42:02.149371 17581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2b2cd90414f403e89ecec8ef250d670: Generated new TSK 0
I20260806 22:42:02.149611 17581 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:42:02.180375 17316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.178639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.16.235.126:45741)
0806 22:42:02.178767 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.178769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.179575 (+ 806us) server_negotiation.cc:407] Connection header received
0806 22:42:02.179709 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.179711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.179723 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.179757 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.180292 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34014: BlockingRecv error: recv got EOF from 127.0.0.1:34014 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260806 22:42:02.181241 17532 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.178331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.16.235.124:42751)
0806 22:42:02.178386 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.178388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.180660 (+ 2272us) server_negotiation.cc:407] Connection header received
0806 22:42:02.180784 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.180786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.180799 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.180832 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.181171 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58396: BlockingRecv error: recv got EOF from 127.0.0.1:58396 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 22:42:02.182101 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.179016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53222 (local address 127.16.235.125:37399)
0806 22:42:02.179070 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.179072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.181498 (+ 2426us) server_negotiation.cc:407] Connection header received
0806 22:42:02.181603 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.181605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.181617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.181647 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.182036 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53222: BlockingRecv error: recv got EOF from 127.0.0.1:53222 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 22:42:02.184846 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.184004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.16.235.125:37399)
0806 22:42:02.184045 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.184048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.184273 (+ 225us) server_negotiation.cc:407] Connection header received
0806 22:42:02.184386 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.184387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.184409 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.184439 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.184795 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53238: BlockingRecv error: recv got EOF from 127.0.0.1:53238 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:42:02.188170 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.187334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53252 (local address 127.16.235.125:37399)
0806 22:42:02.187371 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.187373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.187619 (+ 246us) server_negotiation.cc:407] Connection header received
0806 22:42:02.187739 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.187740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.187753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.187783 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.188119 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53252: BlockingRecv error: recv got EOF from 127.0.0.1:53252 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:42:02.192449 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.191594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53262 (local address 127.16.235.125:37399)
0806 22:42:02.191631 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.191633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.191906 (+ 273us) server_negotiation.cc:407] Connection header received
0806 22:42:02.192007 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.192007 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.192019 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.192050 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.192401 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53262: BlockingRecv error: recv got EOF from 127.0.0.1:53262 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 22:42:02.198762 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.197819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.16.235.125:37399)
0806 22:42:02.197858 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.197860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.198112 (+ 252us) server_negotiation.cc:407] Connection header received
0806 22:42:02.198212 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.198213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.198226 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.198258 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.198620 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53268: BlockingRecv error: recv got EOF from 127.0.0.1:53268 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:42:02.209259 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.208249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53282 (local address 127.16.235.125:37399)
0806 22:42:02.208297 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.208299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.208601 (+ 302us) server_negotiation.cc:407] Connection header received
0806 22:42:02.208732 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.208733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.208747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.208781 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.209195 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53282: BlockingRecv error: recv got EOF from 127.0.0.1:53282 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 22:42:02.227896 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.226766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.16.235.125:37399)
0806 22:42:02.226811 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.226813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.227178 (+ 365us) server_negotiation.cc:407] Connection header received
0806 22:42:02.227288 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.227290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.227303 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.227350 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.227834 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53294: BlockingRecv error: recv got EOF from 127.0.0.1:53294 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 22:42:02.262320 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.261465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53310 (local address 127.16.235.125:37399)
0806 22:42:02.261517 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.261520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.261698 (+ 178us) server_negotiation.cc:407] Connection header received
0806 22:42:02.261790 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.261791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.261803 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.261868 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.262259 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53310: BlockingRecv error: recv got EOF from 127.0.0.1:53310 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:42:02.328631 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.327657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.16.235.125:37399)
0806 22:42:02.327779 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.327782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.327906 (+ 124us) server_negotiation.cc:407] Connection header received
0806 22:42:02.328039 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.328040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.328056 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.328148 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.328556 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53326: BlockingRecv error: recv got EOF from 127.0.0.1:53326 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 22:42:02.459110 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.458169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53338 (local address 127.16.235.125:37399)
0806 22:42:02.458239 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.458242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.458361 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:42:02.458500 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.458501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.458518 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.458590 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.459035 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53338: BlockingRecv error: recv got EOF from 127.0.0.1:53338 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 22:42:02.717192 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:42:02.716520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53346 (local address 127.16.235.125:37399)
0806 22:42:02.716581 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:02.716583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:02.716619 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:42:02.716701 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:02.716702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:02.716713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:02.716758 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:42:02.717108 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53346: BlockingRecv error: recv got EOF from 127.0.0.1:53346 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:42:03.071700 17571 catalog_manager.cc:1259] Loaded cluster ID: 8a87baed082d415495c3df4979ab969a
I20260806 22:42:03.071790 17571 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc: loading cluster ID for follower catalog manager: success
I20260806 22:42:03.072630 17571 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc: acquiring CA information for follower catalog manager: success
I20260806 22:42:03.072950 17571 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72dfe5c133644a36abc87f4972c851bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:42:03.121845 17606 catalog_manager.cc:1259] Loaded cluster ID: 8a87baed082d415495c3df4979ab969a
I20260806 22:42:03.121979 17606 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb: loading cluster ID for follower catalog manager: success
I20260806 22:42:03.122854 17606 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb: acquiring CA information for follower catalog manager: success
I20260806 22:42:03.123180 17606 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a7c3d9da99be4beabe25f16c53b644bb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:42:03.233491 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:03.230605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.16.235.125:37399)
0806 22:42:03.230782 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:03.230784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:03.230893 (+ 109us) server_negotiation.cc:407] Connection header received
0806 22:42:03.230985 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:03.230986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:03.231047 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:03.231130 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:03.231303 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.231827 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.232633 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.232839 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.232919 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:03.233110 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:03.233227 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:03.233313 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:03.233355 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:42:03.233378 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":119,"threads_started":1}
I20260806 22:42:03.237820 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:03.235507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58408 (local address 127.16.235.124:42751)
0806 22:42:03.235622 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:03.235624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:03.235638 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:42:03.235677 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:03.235678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:03.235725 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:03.235806 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:03.235951 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.236445 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.237132 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.237331 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.237382 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:03.237518 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:03.237625 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:03.237688 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:03.237730 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:42:03.237752 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":73,"threads_started":1}
I20260806 22:42:03.243103 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:03.239685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.16.235.125:37399)
0806 22:42:03.239935 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:03.239938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:03.239948 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:42:03.240065 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:03.240066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:03.240079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:03.240159 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:03.240438 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.241148 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.242234 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.242423 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.242538 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:03.242784 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:03.242821 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:03.242852 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:03.242972 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:42:03.243000 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223}
I20260806 22:42:03.243140 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:03.239989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58420 (local address 127.16.235.124:42751)
0806 22:42:03.240031 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:03.240033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:03.240040 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:42:03.240065 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:03.240066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:03.240079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:03.240135 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:03.240282 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.241037 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.242122 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.242418 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.242488 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:03.242694 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:03.242728 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:03.242780 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:03.243024 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 22:42:03.243054 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:42:03.243711 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:42:03.239802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.16.235.126:45741)
0806 22:42:03.240063 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 22:42:03.240065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:42:03.240073 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:42:03.240121 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:42:03.240122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:42:03.240176 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:42:03.240287 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:42:03.241243 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.241729 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.242727 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:42:03.243028 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:42:03.243089 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:42:03.243351 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:42:03.243479 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:42:03.243543 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:42:03.243597 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:42:03.243628 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":82,"threads_started":1}
I20260806 22:42:03.245453 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 17217
I20260806 22:42:03.261205 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 17322
I20260806 22:42:03.271938 17325 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVIGsRX/build/release/bin/kudu with pid 17428
2026-08-06T22:42:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3094 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6184 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287272 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST